About
Sameer Ankalgi — agentic AI systems and GPU inference at Microsoft, Zürich. Agent tooling, evals, vLLM, Kubernetes.
I build agentic AI systems and the infrastructure underneath them — the layer that decides whether a model ships or stalls. I’m an engineer in GitHub AI Engineering at Microsoft in Zürich: agent tooling and evals on one side, GPU-based LLM inference and enterprise AI workloads on Azure on the other.
The work looks like this: building wingman to steer live Copilot CLI sessions from a phone, and pi-bench to grade prompt-injection defenses as a composed stack. Profiling AlphaFold 2 on Azure H100 GPU HPC clusters until large-scale structure prediction was actually viable for a pharma customer. Serving DeepSeek across multi-GPU A100 clusters with vLLM and tensor parallelism, tuned for production throughput rather than benchmark throughput. The gap between those two numbers is most of my job.
A word on the shape of the role: I’m a solutions engineer, which means I don’t run one inference stack — I get dropped into other people’s. Pharma HPC, multi-GPU open-weight serving, enterprise Kubernetes. I’ve seen more ways this breaks than someone who has only operated their own cluster.
I work out of Zürich — Microsoft’s office here and the lake within walking distance of it.
The path a token takes
Almost everything I care about lives somewhere on this diagram.
request
│
┌────▼─────┐ auth · quota · routing
│ gateway │
└────┬─────┘
│
┌────▼───────────┐ continuous batching
│ vLLM scheduler │ paged attention
└────┬───────────┘
│
┌─────┴──────┬──────────┬──────────┐
▼ ▼ ▼ ▼
┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
│ TP 0 │ │ TP 1 │ │ TP 2 │ │ TP 3 │ tensor-parallel shards
└──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘ H100 / A100 · NVLink
└───────────┴────┬─────┴──────────┘
▼
┌────────────┐
│ KV cache │ the thing that runs out first
└────────────┘A demo dies at the model. A production system dies at the scheduler, the cache, or the interconnect — and the person on the pager needs to know which.
How I got here
Twelve years of distributed systems before the GPUs — same failure modes, different hardware:
- Microsoft — agentic AI tooling, GPU inference, and AI workloads on Azure, in GitHub AI Engineering.
- Swisscom — architected classified-grade cloud infrastructure for the Swiss Army, and built GenAI automation for manufacturing SMEs.
- Peech Care — co-founder and CTO of a speech-therapy platform that detected articulation disorders from voice notes. Built the acoustic-feature ML pipeline, led a team of 5, shipped the MVP, raised €150K.
- Daimler (via Devoteam) — moved the connected-car data platform off an IBM Db2 column-store onto bare-metal MongoDB: ~2 TB, 600M+ records, zero downtime, cut over in a 33–38h window at ~4–5K writes/sec sustained. The migration service was Go; the runtime was Kubernetes.
- BCG Platinion and PwC — enterprise delivery, ML pipelines, and the habit of asking what a system is actually for.
- SAP and BookMyShow — HANA backends, and high-throughput transaction queuing under bad connectivity, which is where I learned that latency is a product decision.
The mix — knowing why a thing has to ship and being able to actually build it — is what I optimize for. Full detail on the CV.
What I think about
- Agent infrastructure — session lifecycle, remote steering, fail-safe permission models. wingman is the working prototype: a Go daemon, a zero-knowledge relay, and an iOS app driving real Copilot sessions.
- GPU inference at scale — batching, KV-cache pressure, tensor and pipeline parallelism, and the persistent gap between benchmark throughput and served p99.
- AI safety as engineering practice — containment in the Coming Wave sense: audits, choke points, reversibility. pi-bench is that instinct turned into a benchmark. Paper reviews land under /ai-safety.
- Scalable oversight — oversight has to scale faster than capability, and right now it doesn’t.
I published Reactive Programming Languages — A Survey, which is where the interest in runtime behaviour started.
Recent
- WRK541 — Microsoft AI Tour Zürich — hands-on workshop on real-world code migration with GitHub Copilot Agent Mode
- ABB AI Hackathon with Microsoft
Outside the laptop
Cycling around the Zürichsee in spring/summer, slow miles along the Limmat, occasional travel write-ups.
Find me
If you’re working on agent infrastructure, GPU inference, or the systems behind frontier models and want to compare notes, book a 30-minute chat.