About

Sameer Ankalgi — agentic AI systems and GPU inference at Microsoft, Zürich. Agent tooling, evals, vLLM, Kubernetes.

I build agentic AI systems and the infrastructure underneath them — the layer that decides whether a model ships or stalls. I’m an engineer in GitHub AI Engineering at Microsoft in Zürich: agent tooling and evals on one side, GPU-based LLM inference and enterprise AI workloads on Azure on the other.

The work looks like this: building wingman to steer live Copilot CLI sessions from a phone, and pi-bench to grade prompt-injection defenses as a composed stack. Profiling AlphaFold 2 on Azure H100 GPU HPC clusters until large-scale structure prediction was actually viable for a pharma customer. Serving DeepSeek across multi-GPU A100 clusters with vLLM and tensor parallelism, tuned for production throughput rather than benchmark throughput. The gap between those two numbers is most of my job.

A word on the shape of the role: I’m a solutions engineer, which means I don’t run one inference stack — I get dropped into other people’s. Pharma HPC, multi-GPU open-weight serving, enterprise Kubernetes. I’ve seen more ways this breaks than someone who has only operated their own cluster.

I work out of Zürich — Microsoft’s office here and the lake within walking distance of it.

The path a token takes

Almost everything I care about lives somewhere on this diagram.

      request
         │
    ┌────▼─────┐   auth · quota · routing
    │ gateway  │
    └────┬─────┘
         │
    ┌────▼───────────┐   continuous batching
    │ vLLM scheduler │   paged attention
    └────┬───────────┘
         │
   ┌─────┴──────┬──────────┬──────────┐
   ▼            ▼          ▼          ▼
┌──────┐    ┌──────┐   ┌──────┐   ┌──────┐
│ TP 0 │    │ TP 1 │   │ TP 2 │   │ TP 3 │   tensor-parallel shards
└──┬───┘    └──┬───┘   └──┬───┘   └──┬───┘   H100 / A100 · NVLink
   └───────────┴────┬─────┴──────────┘
                    ▼
             ┌────────────┐
             │  KV cache  │   the thing that runs out first
             └────────────┘
Most production failures live in this plumbing, not in the model.

A demo dies at the model. A production system dies at the scheduler, the cache, or the interconnect — and the person on the pager needs to know which.

How I got here

Twelve years of distributed systems before the GPUs — same failure modes, different hardware:

The mix — knowing why a thing has to ship and being able to actually build it — is what I optimize for. Full detail on the CV.

What I think about

I published Reactive Programming Languages — A Survey, which is where the interest in runtime behaviour started.

Recent

Outside the laptop

Cycling around the Zürichsee in spring/summer, slow miles along the Limmat, occasional travel write-ups.

Find me

If you’re working on agent infrastructure, GPU inference, or the systems behind frontier models and want to compare notes, book a 30-minute chat.