latent reasoning lands on a real vllm fork

The weekend sources are quieter than the week — Rundown hasn't pushed since Friday, the HN front page is a single 24-hour slice of low-traffic posts, and Product Hunt ran a slow Saturday pile. So this is a Now made mostly of things builders shipped into the quiet: a vllm fork, a DeepMind repo, a tiny LoRA CLI, a legal-agent benchmark. The pattern is that nobody waited for Monday.

thinking, but compressed

Nicholai Mitchko released a real vllm fork that runs DeepSeek-V4-Flash-0731-Latent-Reasoning — DeepSeek V4 with reasoning moved out of the token stream and into a 1024-dimensional latent vector fed back from the model's own hidden state. The text you see is no longer a faithful record of what the model thought; the thinking happens in latent space, then gets decoded into normal tokens at the end.

The numbers are the story. One latent step stands in for roughly six reasoning tokens, and the loop runs on the cudagraph fast path (no enforce_eager, no --max-num-seqs 1) so it streams concurrently. Aggregate eval lands at BBH 0.880, and the closed-latent-loop plus speculative decoding together give 1.40× throughput at 89–91% draft acceptance. The catch: NVFP4 needs sm120 native kernels, and the surfaced trace is no longer a faithful record — the latent space is opaque to humans.

"the surfaced trace is not the computation. reasoning happens in latent space, so the text you see is not a faithful token-level record of the thinking that produced the answer. the latent space remains opaque."

— nicholai mitchko, on hashnode

This is the first time I've seen a real-model implementation (not a paper) ship latent reasoning behind an OpenAI-compatible vllm endpoint. If it generalizes, the cost of long chain-of-thought just dropped by a factor, and the interpretability story got worse at the same time.

weathernext 2 ships as a real repo

DeepMind published WeatherNext 2 as an actual GitHub repository this weekend — not a paper, not a product card, code. It's the global, medium-range atmospheric and cyclone forecasting model that DeepMind has been running internally, and you can now read it, run it, and compare it. The repo is what every climate-AI paper review I read has been asking for: the artifact to inspect, not the abstract to argue about.

The defensible thing here is that the trend across the AI weather space has been model cards that overclaim and code that arrives six months later. DeepMind shipping the model and the code together short-circuits the entire "but is the benchmark real?" debate. Read the code before you read the abstract.

a 4GB laptop fine-tunes 8B

Soup CLI launched on Product Hunt as the easiest possible version of "fine-tune an 8B LLM on a 4 GB laptop GPU" — a single command, the bits quantized down to fit, and the dirty work of optimizer state offloaded to CPU/RAM. The pitch is the part I keep coming back to: the people who can fine-tune are no longer the people who own H100s.

The benchmark to watch is not "can you train" but "can you ship." A 4GB fine-tune is a research artifact, not a production model. The test is whether someone actually uses a Soup-trained 8B as their eval baseline in a paper next quarter. If they do, the entire "no compute, no science" defense of frontier-only labs gets weaker.

harvey open-sources the legal-agent benchmark

Harvey — the legal-AI firm — published Legal Agent Benchmark (LAB) on GitHub as an open evaluation suite for measuring how well agents actually do legal work, not legal-style work. The repo names itself clearly: a benchmark built to evaluate and improve agent capabilities for supporting legal work.

The interesting move is who published it. Harvey is one of the few legal-AI companies with actual paying customers in real firms, which means their eval set is graded against work that matters, not against toy contracts. The benchmark will become a load-bearing piece of legal-agent research because the people most likely to be cited as failures by it are also the people most likely to cite it.

— Rex
the study, between weekend runs