- Found the gap. Nemo-RL didn't support Nemotron-Nano-v2-VL — no path to preference-optimize the 12B model.
- Implemented SimPO on Megatron-Bridge from scratch — reference-free RL post-training the framework didn't have.
- Designed the preference data — segmented temporal answers (chosen) vs vague global summaries (rejected), to drive guardrail internalization and fine-grained temporal localization.
Engineering & Production
Projects
Four systems I took from method to production across Pyler and Aiv — the method, the training and inference efficiency, the serving stack, and the proof each one moved the metric.
My contribution
RLHFSimPOMegatron-BridgeNemotron-Nano-12B-VLvLLM
How I built it
- The insight. A VLM that explains its score leaks the decision path — mine enough traces and the latent tree is recoverable.
- A pipeline over 299K traces. Group into 30 buckets (5 categories × 6 scores), a small LLM extracts the decisive signals, another synthesizes them into a yes/no tree (YAML) injected back into the prompt.
- Engineered the trees. Merged equivalent signals into single axes, pulled exceptions to the front as early gates, and collapsed the fuzzy 2-vs-3 / 4-vs-5 bands into a stable 0 / 1 / 3 / 5 scale.
VLM reasoningLLM pipeline299K tracesPrompt engineeringYAML trees
How I built it
- Disentangled defect from background. Diffusion with masked cross-attention and separate defect/background context vectors, trained with a disentanglement loss so a synthetic defect respects its surroundings.
- Controlled placement at inference. DDIM inversion on a normal image, then sample the defect onto a refined mask region.
- Made it affordable on one GPU. Flash-Attention + DeepSpeed + FP16, wrapped in an MLOps pipeline.
PyTorchDiffusersFlash-AttentionDeepSpeedFP16
How I built it
- Two-stage pipeline on NVIDIA Triton. Detection crops the region of interest; a recognition model reads it and matches the DB.
- Profiled, then cut runtime. Triton logs showed model runtime was the bottleneck, not pre/post — so ONNX → TensorRT, warmup, and CUDA graphs.
- Made it weather-proof. Test-time augmentation (rotation, padding) baked into the pipeline.
TritonTensorRTONNXCUDA GraphsDetection + Recognition