ML Engineer · 5+ years in NLP & LLMs
# trained from scratch. no pre-trained weights.
Initialized with a Mechanical Engineering degree, pre-trained on curiosity, and fine-tuned by an unreasonable number of late nights — 5+ years of gradient descent through NLP, LLMs, and generative AI. Currently inference-serving at Apple. I write about ML here so you don't have to train from scratch too.
Apple — multi-agentic systems & LLM research
Notebooks GM, Competitions Master
Dynamic intent detection system
ME degree → ML through sheer will
model.fit(life, epochs=∞, lr=persistence)
Each milestone is a neuron. Experiences flow forward. Lessons backpropagate. The loss is still decreasing.
training_loss.plot()
“You don’t need a low initial loss. You need a good learning rate and the patience to keep training.”
— the philosophy that took me from Mechanical Engineering to Apple
latest
Vanilla RoPE doesn't extrapolate cleanly to sequences longer than its training context. ALiBi does. Yet every production LLM uses RoPE plus an extension trick like YaRN. This is the story of how I discovered that, traced its consequences experimentally, and reconciled it with what production models actually do.
Attention gets all the press, but the rest of the transformer matters just as much. This post traces how feed-forward networks, normalization, residual connections, activations, and embeddings evolved from the original 2017 design to the modern LLM recipe — and why seemingly small changes (removing a bias term, swapping an activation function) compound into models that train faster, scale further, and perform better.
The complete story of Flash Attention: why naive attention is memory-bound despite being 'just matrix multiplies', the GPU memory hierarchy that explains everything, the online softmax trick that makes tiling possible, and how it all composes into the most impactful systems optimization in modern transformers.
A complete architectural dissection of Gemma 4: how hybrid attention (sliding window + global layers), Mixture of Experts, and careful design choices compose into a model that handles 256K tokens efficiently.
Watch my conversations with people who shaped the Indian ML community: