README.txt - Notepad

Nils Matteson

I'm into inference engineering. I like making models faster to start and understanding where the time goes.

I've been working on vLLM through an Inferact-sponsored open-source fellowship with Simon Mo. Most of my time goes into what happens before a model serves its first request: loading weights, compiling code, tuning kernels, and deciding which of that work we can reuse.

I'm interested in GPU memory, caching, and the less obvious reasons a system is slow. I like following a problem through the stack, making a change, and checking whether it helped for the reason I thought it would.

I'm in San Jose, studying CS at Northeastern after finishing at UW-Madison. Outside of that, I make music in Ableton, play guitar, run, and ski when I can.

mail nilsmatteson@icloud.com · code github.com/matteso1 · resume.pdf

inference.txt - Notepad

Lately I've worked on reusable engine snapshots and Python startup in vLLM. Right now I'm looking at MoE kernel tuning in FlashInfer and how to reuse preparation across compatible model servers. A few questions I keep coming back to:

startup	what happens before the first useful response?
model loading	getting weights from storage into GPU memory
reusable state	which work can the next server safely skip?
kernel tuning	finding good choices without repeating the search
measurement	checking why a change made something faster

Current work vLLM FlashInfer

Re-feeding Is Not Replaying.pdf - Acrobat Reader

Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation
sole-author preprint · arXiv:2606.15621 · June 2026

I looked at whether resuming a model's saved decoder state produces the same measurements as feeding the text back in. The experiments compare both against repeated runs, separating replay differences from run-to-run noise. The writeup includes the setup, results, and cases where the distinction disappears.

The paper, experiment logs, and analysis code are public.

Read on arXiv PDF Data + harness

Exploring - C:\nils

Name	Type	Notes
work\vllm	File Folder	startup, model loading, reusable engine state
work\flashinfer	File Folder	MoE kernel tuning and cache reuse
work\sentinel	File Folder	Go log engine, LSM + Raft
work\madison-metro-ml	File Folder	transit ETAs and uncertainty
writing\echo-and-the-engineer.md	Markdown	essay: sycophancy, RLHF, the half-life of a belief
writing\project-gorgon.md	Markdown	speculative decoding at 0.66x of baseline, negative result
writing\madison-bus-eta.md	Markdown	the transit-ETA writeup
writing\wattbot-rag.md	Markdown	RAG benchmarks, 282 questions
refeed-drift.pdf	Adobe Acrobat Document	inference-state replay experiments
agents.txt	Text Document	facts for LLMs