Quantization · failure mechanisms
R&D completeWhy symmetric FP8 can fail
Why does symmetric KV quantization catastrophically break some models? The atlas traces FP8 key failures to learned key bias and incompatible subspace distributions, with controls across model families.
FindingKey and value precision need separate qualification. Retaining BF16 keys is the conservative fallback for fragile models; robust models can still use symmetric FP8.
Learned prefixes · asymmetric precision
R&D completePositive finding · shared cartridges
Keep trained cartridge keys in BF16 and compress values to FP8. On the tested Qwen3-8B cartridges, K16/V8 cuts raw KV bytes by 25% with about 0.03% higher paired evaluation loss.
Serving resultThe fused shared-cartridge path records modest throughput and latency gains over BF16 injection in the controlled runs. Broader deployment qualification remains open.
Decode · exact selection
R&D completeImplementation available · not upstreamed
Use upper bounds to skip vocabulary blocks that cannot win. The research implementation reduces output-head reads and latency for the measured low-batch workloads.
Available to usePublic Triton kernels and artifact-generation tools. Serving-engine integration needs its own numerical and end-to-end qualification.
Storage · workload evidence
Ongoing R&D hubSmall reads · I/O paths · replay
A central tracker for AI workloads that issue small random reads. Follow request sizes, read amplification, direct and buffered I/O, and the role of the page cache.
Ongoing workConnect measured workload evidence with reproducible storage experiments and candidate paths for further analysis.
Storage · CPU · host memory
Workload surveyBulk transfers · host DRAM · folios
Track checkpoints, optimizer offload, object spilling, cache tensors, columnar scans and model loading through host memory.
EvidenceSeparate measured buffer and object sizes from submitted I/O and device requests, then define reproducible host-buffer comparisons.
Architectures · bounded memory
Dense attention, Trellis, DeltaNet and Gated DeltaNet in a matched harness. What fixed-size recurrent state changes, and what the tested scale can establish.
Translation · selected replay · partial recompute
Can one model reuse another model’s cache? Compare span translation, translated selected-key replay, and selective receiver recomputation with their separate evidence boundaries.
Agents · scoped assistance
A framework for small, evidence-producing tasks across security, reliability, research and maintainer work. Reuse agents already in use, with explicit scope and human review.
Operating principleKeep routine tasks bounded, record the evidence, and report urgent security concerns directly to the human owner.
Research automation · evidence
Early pipeline in usePaper discovery · CPU-first validation
Connect the AI Post Transformers paper queue with scoped Chores assistance and reproducible experiments. The goal is to turn relevant papers into tested claims and useful next questions.
First use caseTwo Prefix Integrity Analysis case studies use cheap, Cartridge-based CPU checks to examine adaptive KV-cache methods before model or GPU evaluation.
Fisher information · compression
Bounded findingsResearch roots · quantization
This work led into KV quantization and asymmetric K/V precision. The tested Fisher signals did not significantly predict per-layer KV quantization damage, so direct sensitivity measurements became central.
Scale mattersSome small-model gains did not carry into larger comparisons; other scale-ups remain untested. The collection separates those outcomes.
Cache contracts · correctness
Two separate questions for a cache transform: can the saved prefix still be reused, and does the model still agree?
Training-free · routing priors
Per-block routing priors for selective KV access. Compare the algorithms and the workloads that support their measured results.
Attention · selective access
A pluggable substrate for choosing which KV blocks attention reads. Separate routing policy from the mechanism that gathers the selected state.