Positional embeddings and zero-shot learning using BERT for molecular-property prediction

Table 4 Performance of fine-tuned BERT with relative_key_query PE on various datasets

Task	Data	Sequence	Test loss	Accuracy	Precision	Recall	F1-score
Classification	Malaria	SMILES	0.4858	0.8017	0.8118	0.6866	0.7439
	Malaria	DeepSMILES	0.5203	0.7495	0.7793	0.5622	0.6532
	COVID	SMILES	0.5851	0.7568	0.8484	0.6829	0.7568
	COVID	DeepSMILES	0.4855	0.7568	0.8485	0.6829	0.7568
	COVID-19	SMILES	0.4855	0.7885	0.7881	0.7561	0.7718
	COVID-19	DeepSMILES	0.5171	0.7462	0.7568	0.6829	0.7179
	Cocrystals	SMILES	0.6089	0.6463	0.6102	0.5070	0.5538
	Cocrystals	DeepSMILES	0.6011	0.6402	0.5882	0.5634	0.5755
	BBBP\(^{c_w}\)	SMILES	0.5876	0.7171	0.8039	0.8146	0.8092
	BBBP\(^{c_w}\)	DeepSMILES	0.5422	0.7756	0.8571	0.8344	0.8456
	BBBP	SMILES	0.4679	0.7512	0.7475	1.0000	0.8555
	BBBP	DeepSMILES	0.5592	0.7366	0.7366	1.0000	0.8483
	ClinTox	SMILES	0.4511	0.9262	0.9262	1.0000	0.9617
	ClinTox	DeepSMILES	0.4561	0.9262	0.9262	1.0000	0.9617
	Tox21\(^{c_w}\)	SMILES	0.6579	0.9040	0.9426	0.9560	0.9493
	Tox21\(^{c_w}\)	DeepSMILES	0.6685	0.8419	0.9357	0.8931	0.9139
	Tox21	SMILES	0.3477	0.9394	0.9394	1.0000	0.9688
	Tox21	DeepSMILES	0.2884	0.9394	0.9394	1.0000	0.9688

Bold values denote the best-achieved performance for clarity and emphasis
\({c_w}\) class-weighted function, DeepSMILES the zero-shot learning analysis of BERT

ISSN: 1758-2946