2026-09-25 –, GPN-T Main Room
The boot security community has spent years pushing a measured, verified chain of trust upward from the boot ROM, through firmware, and into the OS. AI inference now runs above that chain, but the trust boundary usually stops where the runtime begins. This talk argues that the KV cache, the working memory of every LLM deployment, is the next surface that belongs inside the root of trust, and shows how to get it there.
In multi-tenant inference the token is the true unit of tenancy. Every tenant’s context occupies scarce GPU HBM, and modern systems tier that cached attention state across HBM, host DRAM, NVMe, and a network fabric to escape the per-GPU eviction cliff. That tiering is what makes dense, long-context inference affordable, but it also scatters confidential state across components that no measured boot flow currently covers. We present LightInferra by Lightbits Labs, a KV cache fabric that serves cached state back across the network, and we focus on the question a boot security audience should care about: how do you anchor that data plane to hardware rather than to software promises.
We walk through hardware-rooted confidential computing for the KV cache: GPU trusted execution environments and confidential compute mode on modern datacenter GPUs, remote attestation of the inference node before any tenant state is admitted, and per-tenant cache encryption with end-to-end integrity monitoring on every cached block, all bound to a hardware root of trust instead of software alone. The goal is an attestable inference node whose chain of trust is continuous with measured and verified boot, so a relying party can verify the platform from the boot ROM up through firmware and into the runtime that holds its data.
We then make the threat model concrete. A shared, tiered KV cache is an attack surface. We examine leaked cached state, inverse projection of cached keys and values back toward plaintext, and silent steering of generation through cache poisoning, and we explain why software-only defenses are insufficient once that state lives on shared fabric and storage. Tying confidentiality and integrity to attested hardware is what closes the gap.
Finally we place this inside a vertically integrated trust stack for AI workloads, connecting boot chain primitives the community already builds, coreboot, UEFI, TPM-based attestation, and owner-controlled boot, to hardware-rooted trust for the inference runtime, protected model weights, and a protected KV cache at the top. Attendees will leave with a mental model of inference as confidential, attestable, multi-tenant state, and a concrete picture of how the root of trust can reach all the way into the working memory of the model.
Arthur Rasmusson is Director of AI Architecture at LightBits Labs, where he works on the KV-cache data plane, Inferra by LightBits Labs - tiering and securing LLM cache state across HBM, DRAM, NVMe, and the network to raise tenant density and extend context length.
His career sits at the intersection of GPU/IO virtualization and large-scale inference: he was previously Engineering Co-Founder of Arc Compute, Principal AI Engineer at Weka and Machine Learning Engineer on Cohere’s Model Efficiency team.
Arthur is a contributor to the open source cache technology and storage ecosystem - author of NVIDIA TensorRT-LLM’s “KV Cache GPUDirect Storage” feature (PR #3209, merged, later moved to NIXL in the NVIDIA Dynamo Stack), Python-Native-libCuFile used in the GPUDirect Storage backend for LMCache/vLLM and creator of Open-IOV.org, a community documenting GPU virtualization, driver, and firmware internals.