Engineering & Production

Projects

Four systems I took from method to production across Pyler and Aiv — the method, the training and inference efficiency, the serving stack, and the proof each one moved the metric.

Pyler · NVIDIA Nemotron Hackathon★ Winner · Track B

SimPO on Megatron-Bridge — for a model it didn't support

Our team's winning entry was a video content-safety VLM on Nemotron-Nano-12B-v2-VL. My piece — the preference-optimization post-training — had no support in the RL tooling.

Team took 1st in Track B; the RL-aligned model held temporal IoU 0.927 while internalizing the guardrails — open-sourced as Megatron-Bridge_RL.

My contribution
  • Found the gap. Nemo-RL didn't support Nemotron-Nano-v2-VL — no path to preference-optimize the 12B model.
  • Implemented SimPO on Megatron-Bridge from scratch — reference-free RL post-training the framework didn't have.
  • Designed the preference data — segmented temporal answers (chosen) vs vague global summaries (rejected), to drive guardrail internalization and fine-grained temporal localization.
RLHFSimPOMegatron-BridgeNemotron-Nano-12B-VLvLLM
Pyler · LLM · Content Safety

Mining Decision Trees from VLM Reasoning

Content-safety scoring — 5 categories × severity — drifted between labelers: the guidelines were flat ladder tables, but the real decision logic lived unwritten in each labeler's head.

category signal? exemption gate? severe? meaningful? weak? no signal exempt severe meaningful weak else 0 0 5 3 1 0

Signal check → exemption gate → severity ladder. Every score traces to a path.

Tree-structured prompts raised labeling consistency and the downstream classifier — and every score is auditable: you can see which node fired.

How I built it
  • The insight. A VLM that explains its score leaks the decision path — mine enough traces and the latent tree is recoverable.
  • A pipeline over 299K traces. Group into 30 buckets (5 categories × 6 scores), a small LLM extracts the decisive signals, another synthesizes them into a yes/no tree (YAML) injected back into the prompt.
  • Engineered the trees. Merged equivalent signals into single axes, pulled exceptions to the front as early gates, and collapsed the fuzzy 2-vs-3 / 4-vs-5 bands into a stable 0 / 1 / 3 / 5 scale.
VLM reasoningLLM pipeline299K tracesPrompt engineeringYAML trees
Aiv · Diffusion · Industrial AD

Background-Aware Defect Synthesis

Real defects are scarce on a manufacturing line — too few, and too narrow in type, to train a detector that generalizes. So I generate them.

Best generation quality (FID / LPIPS) over DFMGAN & AnomalyDiffusion on MVTec-AD & LOCO; raised detector precision & recall on the real line.

How I built it
  • Disentangled defect from background. Diffusion with masked cross-attention and separate defect/background context vectors, trained with a disentanglement loss so a synthetic defect respects its surroundings.
  • Controlled placement at inference. DDIM inversion on a normal image, then sample the defect onto a refined mask region.
  • Made it affordable on one GPU. Flash-Attention + DeepSpeed + FP16, wrapped in an MLOps pipeline.
PyTorchDiffusersFlash-AttentionDeepSpeedFP16
Aiv · Serving · OCR

Real-Time OCR Pipeline for Steel-Plate IDs

Read identifiers stamped on steel plates outdoors — from CCTV and mobile — fast and reliably, under any weather.

Client confirmed throughput and recognition accuracy both rose sharply after rollout.

How I built it
  • Two-stage pipeline on NVIDIA Triton. Detection crops the region of interest; a recognition model reads it and matches the DB.
  • Profiled, then cut runtime. Triton logs showed model runtime was the bottleneck, not pre/post — so ONNX → TensorRT, warmup, and CUDA graphs.
  • Made it weather-proof. Test-time augmentation (rotation, padding) baked into the pipeline.
TritonTensorRTONNXCUDA GraphsDetection + Recognition