ML Systems Desk
Archive
8 articles
- 2026-10-20 Spend Compute Where the Answer Changes: Joint KV-Cache Compaction, Deferred Video Caches, and Load-Aware Sampling
- 2026-10-19 Calling the Target Model Only When Needed: Trajectory Triggers, Bit-Shared Drafts, and Flash Beside HBM
- 2026-10-16 The KV Cache Is Live State: Migrating It at the Edge, and Why a Reader Must Keep Its Own
- 2026-10-15 The Long-Context Bill Has Three Line Items: Cache Memory, Draft Quality, and Tokenization
- 2026-10-14 From Fixed Settings to Measured Ones: Adaptive Draft Configuration, KV Precision Allocation, and Head-Wise Cache Budgets
- 2026-10-13 Moving Fewer Bytes per Token: Replayed SSM State, Load-Adaptive KV Residency, and Calibration-Free 4-Bit Scales
- 2026-10-11 Shrinking What the Accelerator Reads: Preconditioner-Space Rounding, Value-Cache Folding, and Decode-Aware Pruning
- 2026-08-21 Production Inference Architecture: Latency, Speculative Decoding, and Quantization Tradeoffs at Scale