Positional embeddings and zero-shot learning using BERT for molecular-property prediction

Table 7 Performance of fine-tuned BERT with “relative_key” PE on various datasets

Task	Data	Sequence	Test loss	Accuracy	Precision	Recall	F1-score
Classification	Malaria	SMILES	0.4643	0.7787	0.7568	0.6965	0.7254
	Malaria	DeepSMILES	0.5220	0.7557	0.7360	0.6517	0.6913
	COVID	SMILES	0.4617	0.7973	0.8824	0.7317	0.8000
	COVID	DeepSMILES	0.4425	0.8108	0.9091	0.7317	0.8180
	COVID-19	SMILES	0.4526	0.7962	0.7917	0.7724	0.7819
	COVID-19	DeepSMILES	0.4986	0.7808	0.7750	0.7561	0.7654
	Cocrystals	SMILES	0.5827	0.7134	0.6935	0.6056	0.6466
	Cocrystals	DeepSMILES	0.5249	0.7012	0.6410	0.7042	0.6711
	BBBP\(^{c_w}\)	SMILES	0.5651	0.7707	0.8210	0.8808	0.8498
	BBBP\(^{c_w}\)	DeepSMILES	0.5436	0.7902	0.8913	0.8146	0.8512
	BBBP	SMILES	0.2910	0.8585	0.8861	0.9272	0.9061
	BBBP	DeepSMILES	0.2836	0.8780	0.8938	0.9470	0.9196
	ClinTox	SMILES	0.0804	0.9799	1.0000	0.7857	0.8800
	ClinTox	DeepSMILES	0.0421	0.9866	1.0000	0.8571	0.9231
	Tox21	SMILES	0.3964	0.9380	0.9393	0.9984	0.9680
	Tox21	DeepSMILES	0.2282	0.9394	0.9394	1.0000	0.9688

Bold values denote the best-achieved performance for clarity and emphasis
\({c_w}\) class-weighted function, DeepSMILES zero-shot learning analysis of BERT

ISSN: 1758-2946