Teaching a model to match biomedical questions to answers by meaning.
Five students built a semantic similarity model for biomedical question answering. A PubMedBERT teacher scored about 6,000 BioASQ question–snippet pairs and a BioLinkBERT student learned from it. Their adjusted dual-loss training held steady through epochs 4–5, while the baseline started declining after epoch 3. They also found that snippets overlapped across their data splits, which likely inflated their scores, and said so.
Keyword search misses biomedical answers that use different words for the same idea. This team built a semantic textual similarity (STS) system for biomedical question answering using a teacher–student setup. PubMedBERT, pretrained on PubMed text, scored question–snippet pairs built from BioASQ Task 13b (about 6,000 pairs); only 0.66% of negative pairs scored above 0.5, so the teacher separated pairs well. BioLinkBERT was then trained as the student. A baseline using cosine similarity with MSE gave unstable gradients and declined after epoch 3. The team's adjusted dual loss (SmoothL1 plus an alignment loss with adaptive weighting) kept Pearson and Spearman correlations stable through epochs 4–5; extending to epoch 8 showed both declined after epoch 5. They reported that snippets overlapped across their training, validation and test sets, likely inflating the scores, and called for stricter deduplication.
Keywords · semantic textual similarity · biomedical NLP · PubMedBERT · BioLinkBERT · knowledge distillation · BioASQ
Who did this work
MA
Students named here agreed to be featured.
1Why keyword search falls short in biomedicine
Biomedical questions and answers often use different terms for the same concept, so keyword search misses relevant evidence.
Most sentence-similarity models are trained on general text such as Wikipedia. The team wanted a model that measures similarity the way a biomedical reader would.
2A domain teacher and a student model
- Built the dataset. Paired BioASQ Task 13b questions with ideal answers and supporting snippets, and made negative pairs by mismatching them.
- Labeled with a teacher. Used PubMedBERT to score each pair; only 0.66% of negative pairs scored above 0.5.
- Trained a student. Fine-tuned BioLinkBERT to reproduce and refine the teacher's scores.
- Fixed unstable training. Replaced a cosine + MSE loss with SmoothL1 plus an alignment loss and adaptive weighting.
3More stable training, and an honest caveat
| Baseline dual loss | Adjusted dual loss | |
|---|---|---|
| Loss | Cosine similarity + MSE, fixed weights | SmoothL1 + alignment loss, adaptive weights |
| Pearson and Spearman over epochs | Declined after epoch 3 | Stable through epochs 4–5, declined after 5 |
| Training behavior | Oscillating gradients | More stable, delayed overfitting |
4The poster
The team's final poster.
5What this does not show
- Scores are likely inflated by overlap between data splits.
- Labels come from a teacher model, not from human annotators.
- Results are not compared with public STS benchmarks.
6What this work shows
Researcher in Machine Learning and NLP, Concordia University
Researches implicit discourse with language models. Four peer-reviewed papers. Built NLP models for the Government of Québec.
The team built the model and ran the experiments. The Project Lead guided the work and reviewed it.
Natural language processing · Biomedical informatics
A model with a stated data problem and a plan to fix it reads as mature research.
7What the Project Lead observed
The Project Lead wrote feedback on the project at the end of the program.
Written feedback only
This cohort ran before the four-level capability rating was recorded, so this page shows the Project Lead's written feedback without a level.
- Method
- Strong methodological rigor: each architecture and training decision was backed by systematic experiments and clear analysis.
- Understanding
- A solid understanding of model behavior, with limitations addressed thoughtfully.
- Next steps
- Realistic next steps for improvement.
- Overall
- A well-executed example of applied semantic modeling in a biomedical setting.
Source: the Project Lead's and Academic Team's written feedback, 2025.
References
- Gu, Y. et al. (2021). Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare, 3(1).
- Yasunaga, M. et al. (2022). LinkBERT: Pretraining language models with document links. ACL.
- Reimers, N. & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. EMNLP.
- Tsatsaronis, G. et al. (2015). An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics, 16.
Join the AI in Natural Language Processing – Hugging Face Project
Work with an NLP researcher on language models and leave with research you can explain.
