Skip to content

Kernel-Style Machine Learning

Reproducible,
open AI R&D.

We develop and test AI algorithms and systems across models, memory and storage, with open code, measured results and experiments others can repeat.

knlp logo: bridge, surfboard and waves

AI Ecosystem

Follow each topic from research to ecosystem integration: its current stage, the next work, and the upstream dependencies.

LMCache · vLLM · FlashInfer

Asymmetric quantization

UpstreamingKV precision · serving & storage

Separate key and value precision across serving and cache storage. Protect sensitive keys, compress values, and qualify each model.

Current focus25% less persistent KV payload and qualified Qwen3 batch-throughput gains. The tested installation uses pinned public forks; vLLM upstreaming waits for the required FlashInfer support to merge.

Memory · storage · reproducibility

KVTide

Benchmark toolingMemory & storage tiering · comparison

A reproducible I/O harness for tiered KV caches. Compare software NVMe-KV, physical storage and GPU transfer paths.

DirectionBring emerging memory and storage tiering ideas from SPDK, DMA-BUF and AISIO into a common framework for evaluation.

LMCache · vLLM · storage

Speculative prefetching

Re-evaluatingGPU ↔ NVMe · prefetch policy

DMA-BUF + io_uring lets the experimental LMCache GPU-direct path disable the CPU cache tier. That changes the case for prefetching KV state from NVMe into GPU memory.

Why revisitThe earlier retention study did not clear its gate. The direct-storage path creates a new evaluation question; a prefetch benefit is not yet measured.

Shared inference state

KMesh

Prototype evaluationKeyed state · collective reuse

Builds on the speculative-prefetching work in vLLM. KMesh extends the reuse signal from learned prefix caches toward test-time scaling for agentic workflows and chain-of-thought reasoning.

Research directionUse known context and branch relationships to coordinate state reuse and prefetching across related requests.

AI R&D

Mechanisms, measured results and open questions. Each study keeps its controls, limits and negative findings in view.

Decode · exact selection

Certified LM-head decode

R&D completeImplementation available · not upstreamed

Use upper bounds to skip vocabulary blocks that cannot win. The research implementation reduces output-head reads and latency for the measured low-batch workloads.

Available to usePublic Triton kernels and artifact-generation tools. Serving-engine integration needs its own numerical and end-to-end qualification.

Storage · workload evidence

Small random reads in AI

Ongoing R&D hubSmall reads · I/O paths · replay

A central tracker for AI workloads that issue small random reads. Follow request sizes, read amplification, direct and buffered I/O, and the role of the page cache.

Ongoing workConnect measured workload evidence with reproducible storage experiments and candidate paths for further analysis.

Storage · CPU · host memory

Large CPU-initiated I/O in AI

Workload surveyBulk transfers · host DRAM · folios

Track checkpoints, optimizer offload, object spilling, cache tensors, columnar scans and model loading through host memory.

EvidenceSeparate measured buffer and object sizes from submitted I/O and device requests, then define reproducible host-buffer comparisons.

Architectures · bounded memory

Linear attention

Dense attention, Trellis, DeltaNet and Gated DeltaNet in a matched harness. What fixed-size recurrent state changes, and what the tested scale can establish.

Translation · selected replay · partial recompute

Cross-model KV sharing

Can one model reuse another model’s cache? Compare span translation, translated selected-key replay, and selective receiver recomputation with their separate evidence boundaries.

Agents · scoped assistance

Chores

A framework for small, evidence-producing tasks across security, reliability, research and maintainer work. Reuse agents already in use, with explicit scope and human review.

Operating principleKeep routine tasks bounded, record the evidence, and report urgent security concerns directly to the human owner.

Research automation · evidence

AI Auto Research

Early pipeline in usePaper discovery · CPU-first validation

Connect the AI Post Transformers paper queue with scoped Chores assistance and reproducible experiments. The goal is to turn relevant papers into tested claims and useful next questions.

First use caseTwo Prefix Integrity Analysis case studies use cheap, Cartridge-based CPU checks to examine adaptive KV-cache methods before model or GPU evaluation.

Fisher information · compression

FIM-guided R&D

Bounded findingsResearch roots · quantization

This work led into KV quantization and asymmetric K/V precision. The tested Fisher signals did not significantly predict per-layer KV quantization damage, so direct sensitivity measurements became central.

Scale mattersSome small-model gains did not carry into larger comparisons; other scale-ups remain untested. The collection separates those outcomes.

Attention · selective access

KV-cache routing

A pluggable substrate for choosing which KV blocks attention reads. Separate routing policy from the mechanism that gathers the selected state.

More studies & experiments

A little context.
A clearer picture.

Build intuition for the memory traffic, hardware limits and statistics behind the work.

Build with knlp