Table 2 Performances of mapping VO from clinical trials using our proposed approach and other BERT-based models. Acc@n means top n accuracy, n = 1, 2, …, 10
Models | Acc@1 | Acc@2 | Acc@3 | Acc@4 | Acc@5 | Acc@6 | Acc@7 | Acc@8 | Acc@9 | Acc@10 |
|---|---|---|---|---|---|---|---|---|---|---|
BioBERT- v1.1 [65] | 18.2 | 20.9 | 22.7 | 23.6 | 24.5 | 24.5 | 24.5 | 25.5 | 25.5 | 27.3 |
PubMedBERT [62] | 20.9 | 20.9 | 23.6 | 27.3 | 27.3 | 27.3 | 27.3 | 27.3 | 27.3 | 27.3 |
SAPBERT(+ PubMedBERT) [55] | 45.5 | 53.6 | 57.3 | 61.8 | 63.6 | 65.5 | 66.4 | 68.2 | 69.1 | 70.9 |
CTPubMedBERT+ SAPBERT | 45.5 | 50.9 | 55.5 | 59.1 | 60.0 | 63.6 | 63.6 | 64.5 | 65.5 | 67.3 |
SAPBERT + Pub MedMedBERT + VO | 58.2 | 67.3 | 70.9 | 72.7 | 74.5 | 77.3 | 79.1 | 80.0 | 80.0 | 80.0 |
CTPubMedBERT+ SAPBERT + VO | 61.8 | 71.8 | 74.5 | 74.5 | 75.5 | 78.2 | 78.2 | 80.0 | 80.9 | 81.8 |
All-MPNET- base-v2 [66] | 41.8 | 51.8 | 53.6 | 53.6 | 57.3 | 58.2 | 59.1 | 60.0 | 60.9 | 62.7 |
All-MPNET- base-v2-sap-ct-vo | 57.3 | 69.1 | 73.6 | 76.4 | 79.1 | 81.8 | 82.7 | 82.7 | 84.5 | 84.5 |
Ensemble + Score | 60.9 | 71.8 | 74.5 | 74.5 | 74.5 | 77.3 | 78.2 | 80.9 | 81.8 | 81.8 |
Ensemble + Score+ SM-Rule | 70.9 | 80.0 | 82.7 | 82.7 | 82.7 | 83.6 | 83.6 | 86.4 | 86.4 | 86.4 |
Ensemble + Ranking | 62.7 | 70.9 | 74.5 | 75.5 | 78.2 | 79.1 | 80.0 | 83.6 | 86.4 | 87.3 |
Ensemble + Ranking +SM-Rule | 67.3 | 78.2 | 83.6 | 83.6 | 84.5 | 85.5 | 85.5 | 87.3 | 89.1 | 90.0 |
Ensemble + Scale | 60.9 | 70.0 | 75.5 | 77.3 | 80.0 | 83.6 | 85.5 | 87.3 | 88.2 | 88.2 |
Ensemble + Scale+ SM-Rule | 71.8 | 80.0 | 82.7 | 83.6 | 84.5 | 85.5 | 88.2 | 89.1 | 90.0 | 90.0 |