Building visual SLAM from scratch, and finding out where it breaks.
Atsuki Abe and four teammates implemented a graph-based visual SLAM pipeline, with visual odometry, loop closure and g2o pose-graph optimization, and tested it three ways: on the KITTI benchmark, on 11 photos of ArUco markers, and live with an Intel RealSense RGB-D camera. On the marker set, optimization pulled a misaligned map into a coherent one within 19 iterations, but the team reported that their ground truth was not precise enough to measure accuracy.
Simultaneous localization and mapping (SLAM) lets a robot estimate where it is while building a map. This team implemented graph-based visual SLAM from scratch: FAST features with Lucas–Kanade tracking for odometry, essential-matrix or PnP pose recovery, loop-closure checks against earlier frames, and pose-graph optimization in g2o with a Levenberg–Marquardt solver. They tested it on the KITTI odometry benchmark, on 11 iPhone photos of 0.1016 m ArUco markers, and as a real-time pipeline on an Intel RealSense RGB-D camera. On KITTI, translation tracking followed the ground truth but rotation error grew. On the marker set, odometry with both ORB and SIFT drifted; graph optimization made the trajectory and landmarks consistent by iteration 19, but printed markers were detected poorly, so the ground truth itself was imprecise. The live demo ran on a CPU and worked best in textured, static scenes with moderate motion.
Keywords · visual SLAM · pose-graph optimization · g2o · KITTI · ArUco markers · RGB-D
Who did this work
- On this project
- Explained the graph-based SLAM method clearly in the team's presentation.
Students named here agreed to be featured. 4 other team members are not named. Role notes come from the Project Lead's feedback.
1Why build SLAM yourself
Robots that move through unknown spaces need to estimate their own position while mapping the space around them. Libraries do this well, but they hide where errors come from.
The team built each part themselves to see how odometry errors and observation noise interact, and how much optimization can repair.
2Front end, loop closure, back end
- Tracked motion. FAST features with Lucas–Kanade optical flow between frames, then essential-matrix (RANSAC) or PnP pose recovery.
- Detected loops. Every five frames, compared the current frame with earlier ones using SIFT and brute-force matching.
- Optimized the graph. Built a pose graph and solved it with g2o's sparse optimizer and a Levenberg–Marquardt solver.
- Tested three ways. KITTI odometry (with GPS/IMU ground truth), 11 photos of ArUco markers, and a live RealSense RGB-D pipeline with re-localization.
3What worked in each setting
| Setting | What happened |
|---|---|
| KITTI benchmark | Trajectory followed the ground truth; rotation error grew over the run |
| ArUco marker photos (11) | ORB and SIFT odometry both drifted; optimization made the map consistent by iteration 19, but not precise |
| Live RGB-D camera | Ran on a CPU; worked in textured, static scenes; struggled with low texture, fast motion and noisy depth |
4Selected slides
Slides from the team's final presentation.
5What this does not show
- No standard accuracy metric (such as absolute trajectory error) is reported.
- The marker test used only 11 photos; denser capture or video is listed as next step.
- Bundle adjustment before optimization was planned but not tested in the time available.
6What this work shows
Researcher in Robot Perception and Manipulation
CEO of ROBOX and advisor at XYZ Robotics. MIT EECS PhD. Best Paper Finalist at IROS and ICRA. First place in the Amazon Robotics Challenge stowing task (2017).
The team built and tested the system. The Project Lead guided the work and reviewed it.
Robotics · Computer vision · Autonomous systems
A SLAM system built from scratch, with honest results, shows you understand what libraries hide.
7What the Project Lead observed
The Project Lead wrote feedback on the project at the end of the program.
Written feedback only
This cohort ran before the four-level capability rating was recorded, so this page shows the Project Lead's written feedback without a level.
- Understanding
- Strong grasp of core SLAM concepts and the ability to implement a complete system end to end.
- Experimental depth
- Built and evaluated several SLAM variants across a public benchmark, self-collected data and a real-time setup.
- Communication
- Atsuki explained graph-based SLAM clearly; the team used AR tags thoroughly to validate behavior.
- Execution
- Delivered a live on-site demonstration with the provided camera on a compressed on-campus timeline.
Source: the Project Lead's and Academic Team's written feedback, August 2025.
References
- Geiger, A. et al. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. CVPR.
- Kümmerle, R. et al. (2011). g2o: A general framework for graph optimization. ICRA.
- Grisetti, G. et al. (2010). A tutorial on graph-based SLAM. IEEE Intelligent Transportation Systems Magazine, 2(4).
Join the AI and Robotics for Mobile Robot Manipulation – Boston Dynamics Project
Work with a robotics researcher on localization and mapping and leave with a system you understand end to end.
