Upload a brain MRI. Get the tumor outlined and its volume measured.
A team of three students adapted a pretrained vision model to read brain MRI scans and turned it into a working web tool. On 73 patients the model had never seen, its tumor outlines matched radiologists' at 0.848 on a 0–1 overlap scale.
Measuring a brain tumor still depends on experts tracing it by hand, slice by slice. This team adapted an ImageNet-pretrained hybrid CNN–Vision Transformer to four-sequence brain MRI, trained it on a patient-level split stratified by tumor volume, and evaluated it once on 73 held-out patients. Whole-tumor overlap with expert labels reached a Dice score of 0.848, matching validation, compared with 0.48 for a model trained from scratch. The model was deployed as a web tool that segments, measures and exports tumor volume. Results come from a public research dataset and are not intended for clinical use.
Keywords · brain MRI · tumor segmentation · volumetry · transfer learning · Vision Transformer · held-out evaluation
Who did this work
- Concentration focus
- AI + Health Tech
- Concentration focus
- AI + Health Tech · AI + Business & Finance
- Concentration focus
- AI + Business & Finance
Students are not named. Year of study is as of fall 2026. Direction and goal come from each student's own program plan.
1Why measure tumors automatically?
To track a brain tumor, a clinician traces its border slice by slice across dozens of MRI images. It is slow, and two experts rarely draw exactly the same line.
In drug trials, a change in tumor volume is one of the clearest signs of whether a treatment works. A tool that outlines tumors consistently turns "visible on a scan" into a number that can be compared over time.
2From raw scans to a working tool
- Audited the data first.484 patients from the Medical Segmentation Decathlon brain tumor task [1], each with 155 image slices and four MRI sequences. Tumor sizes ranged from 7 to 318 mL, so the team made sure every group would contain small, medium and large tumors.
- Split patients, not images.Each patient appears in only one group. The test group was locked away until the very end.
- Built a from-scratch baseline.Starting from a transformer script the Project Lead provided, a teammate trained a model from zero. It reached only 0.48 overlap, which set a clear bar to beat.
- Adapted a pretrained model.The team took a hybrid ResNet–Vision Transformer [2] pretrained on millions of everyday images and modified its input layer to read all four MRI sequences at once.
- Shipped it as a web tool.The team deployed the model on Hugging Face: upload a scan, inspect the outline in 2D and 3D, read the volume, and export the result.

3Pretraining made the difference, and the result held on new patients
| Model | Evaluated on | Mean Dice [3] | Median volume error |
|---|---|---|---|
| From-scratch ViT baseline | Validation | 0.48 | — |
| Pretrained CNN–ViT | Validation (72) | 0.848 | 8.2% |
| Pretrained CNN–ViT | Held-out test (73) | 0.848 | 8.7% |
| Four-label variant | Held-out test, whole tumor | 0.845 | 8.9% |

4What this does not show
- It is a research demo, not a clinical tool.
- All results come from a public research dataset (Medical Segmentation Decathlon) and have not been clinically validated.
- Sub-region outlines are not yet reliable; volume should be read from the whole-tumor result.
5Selected slides


6What this work shows the team can do
Researcher in Medical Imaging and Machine Learning
Senior Imaging Scientist at Novartis. MIT PhD in ultrasound imaging and medical robotics. Co-founded two medical startups.
The team ran the experiments and built the model and the web tool. The Project Lead met with the team weekly and provided a starting transformer script.
Medical imaging · AI · Biomedical engineering
A working tool reviewers can understand at a glance makes a strong portfolio piece.
7What the Project Lead observed
The Project Lead evaluates every student on a project in two parts: written notes on how the work was framed, decided, tested and bounded, then one overall level out of four. The notes below describe the team's project. Individual ratings are not published.
System-Level Insight
- 1Foundational Application
- 2Applied Reasoning
- 3Independent Problem Solving
- 4System-Level Insight
Level 4 of 4: anticipates broader implications, risks or downstream effects, and connects technical decisions to real-world impact.
- Framing the problem
- Audited the dataset for slice-level leakage before training, and built the evaluation around a frozen, patient-level split stratified by tumor volume. Set whole-tumor segmentation as the clinical output, and scoped the work end to end, from raw MRI intake to a 3D volume report.
- Decisions and trade-offs
- Adapted a hybrid ResNet-50–ViT, pretrained on everyday photos, by changing its input stem and positional embeddings to take four-channel MRI. Built training and 3D held-out evaluation for both the binary and four-label versions, and chose cloud GPU inference on Hugging Face so the web tool could read scans, show them in 2D and 3D, measure volume and export masks.
- Evidence and testing
- Scored the model on whole held-out patients in 3D rather than on single slices, using patient-level Dice and the spread of volume errors. Checked results across all four MRI sequences, and added an option in the web tool to compare a prediction with an expert label.
- Limits and risk
- Kept each patient in a single split to rule out cross-slice leakage. Designed the tool to show its work, with 2D and 3D views, an explicit volume figure and a downloadable mask, so each result can be audited.
Source: the Project Lead's written evaluation, August 2026.
References
- Antonelli, M. et al. (2022). The Medical Segmentation Decathlon. Nature Communications, 13, 4128.
- Dosovitskiy, A. et al. (2021). An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR 2021.
- Dice, L. R. (1945). Measures of the Amount of Ecologic Association Between Species. Ecology, 26(3), 297–302.
Join the AI and Computer Vision in BioTech – Novo Nordisk Project
This outcome came out of this project. Work with a medical-imaging researcher on a real problem, and leave with something you can open and show.
By clicking "Accept", you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.