Fig. 2

Visual representation for a molecule in the TDC dataset of the comparison between the most relevant eigenvectors of the Attention Rollout matrix from a model pretrained on atom-level QM properties and the low-frequency eigenvectors of the graph Laplacian associated to the molecular structure. Each eigenvector is a function defined on the graph nodes, and hence the color scale used here corresponds to \({|{a_i}\rangle }\) for the eigenvectors of the Attention Rollout matrix \(\tilde{A}\) with eigenvalue \(a_i\) and to \({|{l_i}\rangle }\) for the eigenvectors of the graph Laplacian L with eigenvalue \(l_i\)