July 1, 2026
- Work on registering the point cloud to the 3D model
- Probably not able to generate a 3D model through the point cloud / .ply
June 6, 2026
Background research: dive deep into the field and understand the problem and the challenges.
- Design and print mechanical parts to install the camera and LED together. (Andy)
- Use OpenCV to calibrate endoscope orientation. (Andy)
- Will humans detect LEDs turning on/off at high frequency?
- Write a protocol to collect 2D endoscope data for study: DataCollectionProtocol — done
May 18, 2026
LuminaBone Project
- Draft a paper for publication.
- Find a conference to submit.
May 17, 2026
Action Items
- Utilize a photometric stereo algorithm to calculate object surface normals and generate depth images, including 3D point cloud data. Implementation code has been uploaded to GitHub. Refer to the Photometric Stereo Depth Reconstruction report for technical details. (Andy Ding) — done
- Define the application for the LuminaBone project. While the current photometric accuracy (2–3 mm) is insufficient for high-precision bone registration or motion tracking, it is suitable for instrument orientation guidance and environmental awareness to assist surgical navigation. (Jayson Ding) — done
- Limitations: wet bone surfaces may impact model reflection accuracy (1–3 mm range). However, this level of accuracy remains a significant improvement for clinical navigation compared to current non-navigated procedures.
- Supporting documentation: CavityScan_Technical_Protocol_v0.1, Depth_Technology_Assessment_Spine_Endoscopy, and How to Present the Depth Map to the Surgeon
- Acquire a series of stereo images using spine bone models for further testing. (Jayson Ding)
- I will need JK’s support to record this video. The data can be collected under the spine navigation setup environment. One endoscope will be attached to a probe, and three LEDs will be installed on the endoscope — on the left, right, and top. These LEDs need to be turned on and off as required. Once the setup is ready, the video should be recorded for post-processing.
- Data collection: endoscope spatial position, bone location; sequentially turn on the three LEDs and take 3 static images for processing.
- Design a jig to attach three LEDs and the endoscope together. The endoscope we ordered comes with an LED; you just need to design a jig to hold 4 endoscope cameras — one is used for the camera, the other will be used as a light source. We are using the endoscope ordered from Taobao, similar to this one but with two cameras, straight and 90-degree angles. (Andy Ding)
The source links a reference drawing here — V1000 camera Drawing v2 (PDF) — not mirrored on this page.
May 16, 2026
Traditional bone registration is often time-consuming and error-prone due to the manual selection of optical markers and CT fiducials. The Lumina Bone project addresses this by utilizing sensor-generated depth maps to register with pre-operative CT images, facilitating efficient registration and real-time navigation.
Generating depth imaging via the photometric stereo approach.
The source includes concept images here illustrating the photometric-stereo depth imaging setup, not mirrored on this page.
Project Proposal: 2026-summer-intern_project_proposal