Business constraints, research questions, shipped systems.
Each case starts with an outcome that matters, works down through the technical
uncertainty, and ends with something measurable, reliable, and useful in a real workflow.
These case studies deliberately describe the technical decisions and outcomes without
exposing proprietary data, customer material, or internal system details.
Benchmarks
Production Results
Benchmarks are polite. Production data is more honest.
The benchmark gets the system out the door. The anomaly decides what I investigate next.
Agent evaluation
Making agent failures diagnosable
2025 – Present
Core question
How do you tell whether an agent failed at perception, generation, evaluation, or verification?
The constraint
Aggregate scores could show that an agent failed, but not which component or behavior caused the failure. The evaluation system needed to catch weaknesses before release and turn failures observed in production into specific, actionable diagnoses.
What shipped
Built the team’s evaluation harness for pre-production quality gates and post-production diagnosis, making failed runs easier to reproduce, classify, and route toward the right system improvement.
Investigation → system
01 Built automated grading, adversarial probing, and red-teaming workflows with Promptfoo, Garak, and DeepEval.
02 Implemented a custom BinEval-style decomposition that separates perception, generation, evaluation, and verification failures.
03 Adapted Genie into a synthetic document Q&A pipeline tuned to generate questions that resemble real user intent.
Paper → implementation
Turning documents into evaluation data
Genie provides the data-generation loop. The implementation adapts it to produce
source-grounded questions that behave more like the questions people actually ask.
Clean and transform source material into content that can support task-specific examples.
In this implementation
Engineering documents become bounded source contexts, so every generated example begins with evidence rather than a free-form prompt.
Output
Grounded content units
In the paper
Generate task-specific examples from the prepared content.
In this implementation
The generation stage is tuned toward questions that resemble real user intent, not the overly neat phrasing common in synthetic datasets.
Output
Human-like Q&A candidates
In the paper
Remove examples that are low quality, unfaithful, or insufficiently grounded in the source.
In this implementation
Candidate pairs pass three gates: source grounding, user-like intent, and evaluation utility. Plausible wording is not enough if the answer is unsupported or the question would not reveal useful behavior.
Output
Grounded, usable pairs
In the paper
The paper produces content-grounded datasets for downstream tasks.
In this implementation
Retained examples become repeatable quality gates before release and regression cases when production exposes a new failure mode.
Output
Pre- and post-production eval set
Select a stage to inspect the handoff. The 100 → 65 → 35 → 15 gate counts illustrate
progressive filtering, not reported production metrics; source documents and evaluation data are not shown.
Document intelligence
Turning difficult documents into reliable structured data
2025 – 2026
Core question
How do you extract schema-constrained data from difficult engineering documents without allowing silent errors to propagate?
The constraint
The document mix included handwriting, low-resolution scans, complex layouts, and domain-specific symbols. Outputs also had to conform to a downstream schema, so a plausible answer wasn't enough when silent errors could propagate through the workflow.
What shipped
On a held-out set of 100 documents spanning handwriting, low-resolution scans, and complex layouts, field-level extraction accuracy went from 32% with the previous LLM-based pipeline to 80%, at 2.1 pages per second on an L40S with schema-valid JSON on 100% of runs. The verification database changed the reviewer’s job from transcribing fields to correcting them.
First attempt, and why it did not hold
The pipeline started as a single multimodal LLM: PDF in, structured output out, one pass.
It reached 32% field-level accuracy, and the number was the smaller problem. Reading and extraction happened in the same pass, so a wrong field could mean the model misread the handwriting, or read it correctly and assigned it to the wrong key — with no way to tell which. It also failed quietly, returning fluent, schema-shaped, confidently wrong values. Splitting the stages apart, OCR first and then a constrained LLM, made every failure attributable to one of them. That is what turned an accuracy problem into a fixable one.
Investigation → system
01 Benchmarked open-source vision-language models and paired LightOnOCR2 with a prompt-engineered LLM stage.
02 Produced schema-constrained JSON with validation designed around the downstream application workflow.
03 Built a verification database that captures human corrections and feeds them back into the extraction loop.
3D perception
Learning from severely imbalanced point clouds
2025
Core question
How do you improve minority-class learning without sacrificing the rest of a large outdoor scene?
The constraint
The enterprise dataset contained approximately 3 million labeled points, including transmission towers and conductors. Utility point clouds are dominated by common classes while operationally important classes can be rare, so overall accuracy can hide the failures that matter.
What shipped
On held-out validation data, the operationally important minority class reached 92% IoU, against 98% IoU on the majority class. Overall accuracy also reads 98%, but on a point cloud this imbalanced that number is close to what predicting the majority class alone would score — the minority-class IoU is what the model was tuned and judged on. The work also shortened the iteration loop, and I mentored an intern through the modeling and data-pipeline work.
Investigation → system
01 Built on RandLA-Net and KPConvNeXt with focal loss and class-balanced re-weighting.
02 Combined farthest-point and inverse-frequency sampling with targeted oversampling, tile-level rebalancing, and minority-class augmentation.
03 Improved the training, preprocessing, and annotation loops through caching, batched I/O, and a redesigned labeling workflow.
Those ~3 million points did not label themselves. The intern and I spent two months annotating them, running on our last two brain cells.
From reconstruction papers to a production pipeline
2024 – 2025
Core question
What has to change when a promising 3D reconstruction method meets real images, runtime constraints, and deployment infrastructure?
The constraint
High-fidelity reconstruction from hundreds of high-resolution indoor images was useful only if processing time and rendering speed fit the operational workflow.
What shipped
On comparable scenes, enabled 30 FPS rendering and reduced processing for complex scenes to under 60 minutes—an 80% speed improvement over the earlier architecture.
Investigation → system
01 Built a structure-from-motion pipeline with COLMAP and OpenSFM for 500+ image scenes.
02 Integrated Gaussian Splatting and 2DGS for photorealistic real-time rendering.
03 Moved to a GLOMAP-based architecture and containerized the pipeline for repeatable deployment.
Active experiment
In progress
On-device 3D capture with LiDAR priors
2024 – Present
Core question
Can mobile depth and pose estimates make field capture simpler while improving downstream reconstruction?
The constraint
Manual survey workflows are cumbersome, while image-only reconstruction can struggle without useful geometric priors.
What shipped
The app is still in development. The current goal is a usable capture workflow and better depth priors—not a polished demo claim before the system has earned it.
Investigation → system
01 Fuse ARKit SLAM with iPad LiDAR depth for in-field point-cloud capture.
02 Explore on-device and cloud-based Gaussian Splatting reconstruction paths.
03 Design the capture flow around the constraints of the person collecting the data.
Earlier work
Coursework and foundations
University projects from 2021–22. Kept partly because the fundamentals hold up, mostly because deleting them sounded like work.
+
Computer Vision
Structure from Motion
2022
A from-scratch SfM pipeline covering geometry, triangulation, and bundle adjustment.
