ML Systems Engineer · Distributed Training · LLM Infrastructure · Multimodal Systems
I build systems that work at scale -- and try to understand why they fail when they do.
I work in the space between clean research ideas and the messy reality of clusters that fail, data that drifts, and models that need to stay honest in production.
Things I care about technically
- Large-scale pre-training infrastructure -- MoE routing, fault-tolerant checkpointing, tensor/pipeline parallelism
- LLM safety and observability -- keeping models (and the agents built on them) honest at inference time
- The hardware-software boundary: SIMD, CUDA, kernel-level optimization
- Novel architectures worth deploying, not just benchmarking
Things I care about less technically
- Code that impresses interviewers but breaks on week two
- Benchmarks that only win on synthetic data
- Documentation that describes the happy path and nothing else
These are small but high-signal fixes in core infrastructure:
| Project | Status | What was fixed | Link |
|---|---|---|---|
| Triton | Merged | Fixed NaN handling in tl.argmin / tl.argmax so interpreter matches JIT behavior |
PR #10699 · write-up |
| TensorFlow | Merged | tf.experimental.numpy.swapaxes now raises a clear error on out-of-bounds axis instead of silent normalization or opaque XLA errors |
PR #122544 · write-up |
| Megatron-LM | Merged | Fixed crash in get_grad_norm_fp32 when gradient list is empty (common with frozen layers / tensor parallelism) |
PR #5530 · write-up |
I'd rather have a few things that are real than many that just look good on a profile.
| Project | What it is | Where it actually stands |
|---|---|---|
| KANX | Production-oriented Kolmogorov-Arnold Networks library (PyTorch + TensorFlow + ONNX) | pip install kanx ·  · Colab · write-up · DOI · latest release v0.2.1 |
| Composed-MoE-Engine | Fault-tolerant MoE training runtime with fused Triton router, 4D parallelism, elastic recovery, and async checkpointing | 80.1× (T4) / 58.7× (A100) kernel speedup · 348 tests · three hardware-only bugs documented · verified at 2-rank / single-GPU · multi-node still open |
| filing-qa | Question answering over financial filings with cited answers (hybrid dense+BM25, chart extraction, numeric guardrail, AST-sandboxed calculator) | FastAPI + CLI + Helm · 119-question golden set · no fabricated numbers on the set · ~30% still miss at retrieval |
| guardrail-rs | Zero-Python LLM security reverse proxy in Rust (prompt injection, PII redaction, policy enforcement) | Sub-ms overhead on the default path · fails open by design · five crates on crates.io · Colab · write-up |
| FlashSpec | Lossless speculative decoding for Hugging Face causal LMs with static-cache CUDA-graph fast path | pip install flashspec ·  · v0.3.2 · ≥1.5× over compiled HF generate on T4 (Qwen2.5, batch 1) · still Alpha · earlier bandit/Triton path retained as experimental |
| simd-ml-kernels | Hand-written AVX2/AVX-512/VNNI microkernels for core inference arithmetic | Runs Llama-3.2-1B end-to-end, token-identical to Hugging Face in FP32 · 1.02–1.69× llama.cpp on the same cores in measured configs · still loses to OpenBLAS/PyTorch in several cases · README lists every one |
Not everything is a shipped tool -- some of it is just a question I wanted a real answer to.
- Does fine-tuning break a transformer's copy mechanism? Write-up →
- rlpost -- LLM post-training from scratch (SFT, reward model, PPO, DPO); parity-tested against TRL
- production-vlm-engineering -- Reproducible pipelines for modern multimodal vision systems
- mcp-reliable -- Runtime observability for MCP servers (health, per-tool latency, output-shape drift)
- PromptCanary -- Detects silent behavioral drift in LLM providers (
pip install promptcanary)
Training & inference -- PyTorch TensorFlow Triton ONNX TensorRT FSDP2
LLM -- Transformers PEFT vLLM FastAPI
Infra -- NCCL Kubernetes Helm Ray
Observability -- Prometheus Grafana OpenTelemetry
Low-level -- C++ CUDA SIMD pybind11
|
|
Algorithms are how I warm up. Systems are where I live.
Most of my interesting work happens in private repositories. This GitHub is a public window, not the full picture.
The repositories here are written to the same standard I use privately: tests, type checking, CI, real (if limited) benchmarks, and documentation that tries to admit what doesn’t work yet.
My path into this wasn’t linear. It started in mechatronics -- building systems with real sensors, actuators, and control loops that fail in ways no unit test catches -- and later working on ML systems inside industrial control and safety environments, where a silent failure has real consequences. That background is the root of the reliability focus you see in these projects.
- Working on: the MoE engine’s chaos scenario A -- sudden node failure under expert resharding (still recovering ~85% of the time; Gloo race remains)
- Reading: the Megatron-LM codebase and the FlexAttention paper
- Thinking about: whether MFU tracking gives enough signal to catch silent training degradation early
Outside of work I’m usually reading something I don’t fully understand yet, listening to music that has no business being that good, and occasionally wondering if the model actually converged or if I just got lucky. I like working with people who say “I don’t know” without embarrassment and argue about architecture in good faith.