The Sub-$1,000 MoE Cluster: Serving a 70B Mixture-of-Experts with Speculative Expert Paging

How asynchronous NVMe streaming, router logit lookahead, and tiered ring buffers push production-grade inference out of commodity hardware.

Content

H A R S H H A A6 days ago

Scaling Beyond a Single Box: Engineering Multi-Node Distributed LLM Serving for Production

How tensor parallelism, 200 Gb/s RoCE, and low-level runtime forensics turn raw multi-GPU hardware into a rock-solid, 1-million-token AI serving platform.

3,721200

H A R S H H A AAug 14, 2026

AI Token Cost: What Is a Token — and How Every Model Games the Count

Complaints about opaque AI pricing are flooding social media. Which ones hold up to scrutiny and which are just users' overactive imaginations? How do models slip past their own published rate cards? How do usage limits actually work, and why do companies benefit from keeping the explanations as vague as possible?

10,059382

H A R S H H A AMay 2, 2025

End-to-End DevOps with Azure and GitHub Actions: A Developer's Journey

In this blog, I share how to set up a simple yet powerful CI/CD pipeline using GitHub Actions and Azure App Service. From automating deployments to securing credentials and running tests, this guide walks through the end-to-end DevOps workflow—perfect for developers looking to streamline cloud deployments with minimal setup

56,3063,943