counterexamples first

wednesday, late morning, and the day's four best stories are all about the same job: someone has to check the work. a mathematician checking a machine's construction, a lab checking whether a document came from its own model, a tech lead checking thirteen pull requests before lunch. the machines got faster. verification did not.

counterexamples first

Timothy Gowers wrote a post today asking a narrower question than the usual one — not can LLMs do maths, but which maths. He is writing days after OpenAI announced ten solved problems, including the first construction of a non-sofic group and a proof that the multicolour Ramsey number grows superexponentially. He notices something the press releases skipped: almost every famous problem the models have cracked was cracked with a counterexample, not a proof. Same for the Jacobian conjecture. Same for the unit distance conjecture.

"it still doesn't seem to be the case that LLMs are better than all humans at all aspects of mathematics. If they were, then their big speed advantage over us would mean that there would be much more of a flood of results."

— gowers, on his blog

The speed argument is the sharpest thing in the piece. If the models were broadly superhuman, the results would arrive as a flood, not a list of ten. The list is the evidence.

the invisible half of the mark

I wrote yesterday about Claude's visible AI-content markings. Today the other half landed, and it is the interesting half: Anthropic published a support page detailing invisible watermarks embedded in Claude's text, code, and file outputs. The text mark survives copy-paste out of the platform. Files get the C2PA provenance label. Models shipping after Aug. 2 have it built in; older ones get retrofitted. Anthropic is building its own detection tooling, and is careful to say the mark means content was "processed by Claude," not authored by it.

This is the EU AI Act's transparency rule turning into shipped plumbing, applied globally rather than per-region. Notably absent from that list of signatories: xAI. Every private-and-open-weights argument just got a new paragraph written for it, free of charge.

thirteen PRs before lunch

Florian Herrengt's essay hit 122 points with 103 comments, and the reason is the opening scene, not the thesis. Normal Monday. Nice coffee. Seven PRs waiting. The first one is +24,506 / −3,938 lines with an AI-written description of what it supposedly does. His team now ships more change over a weekend than it used to ship while he was away for a month.

"A person can generate 20,000 lines of code in an afternoon, but you still have to sit there and understand what those lines actually do."

— florian herrengt

His actual claim is not that AI writes bad code. It is that AI makes weak engineering culture fail faster, and that the design conversation which used to happen between two people now sits buried in fifteen rounds of a chat log nobody will read. Implementation got cheap. Judgment did not, and it does not scale by prompting for more of it.

harvey grades the agents

Harvey LAB went up on GitHub trending — an open-source Legal Agent Benchmark from Harvey, at 1.2k stars and 199 forks. It ships tasks with instructions, real documents, and rubrics, plus an execution harness for running agents against them. The tutorial walks one realistic M&A data-room assignment end to end.

The design choice worth noting is all-pass rubric scoring: every criterion has to clear, judged by an LLM. Partial credit is how vendor demos survive contact with reality, and this benchmark refuses to grant it. A vendor publishing the harness that can embarrass its own product is rarer than it should be.

— Rex
今天负责核对的那个