watermarks, books, and red agents

monday morning, and the loudest story in ai is, again, a quiet one. anthropic announced a way to embed invisible fingerprints in the words claude chooses. john gruber read the technical doc. then he wrote a 4,000-word essay that ends with the word poisonous three times. that's the day's gravity well.

dario logs on

anthropic's ceo went on x yesterday to push back on a thread between investor gavin baker and sholto douglas. baker had argued dario's safety talk is backfiring — handing ammunition to anti-data-center campaigns, and that the "lock it down vs. open it up" framing is a false choice. amodei replied that the only path back to public trust is by actually curing cancer, not by softening the message. his concrete ask: that anthropic's proposals should slow the big labs while sparing the small ones. that's the kind of answer that only makes sense if you already believe the diagnosis.

the watermark is the message

gruber's piece is the strongest single argument against semantic watermarking i have read. his core claim: anthropic's own document says the marks "don't change the meaning, quality, or readability." by definition the algorithm has to slightly bias claude toward worse word choices some of the time. so the claim is a contradiction in terms. the strongest line in the post is the last one.

"Secrets are the poison here. When only Anthropic holds the secret keys that both produce the watermarking and perform the probabilistic detection of those marks, we're all left to wonder."

the regulatory pretext is the eu's code of practice on transparency of ai-generated content. anthropic is rolling the system out globally, not just in the eu, because they say they cannot scope it regionally. gruber is not buying that. his read: a company weeks away from a $2t ipo is technically incapable of compliance-by-region? sure.

the cleanest takedown of the law itself comes from james padolsey, whose interactive essay on how watermarking works gruber's post links to:

"To make assistance suspect only once the tool becomes capable enough to compose a whole sentence is not a principled boundary. It is a moral premium placed on difficulty itself."

the meta-lesson: the way you mark the text is the message, and you cannot promise the marking does not mark it.

amazon is cutting bindings off rare books

404 media planted a tracker in a rare book they suspected would be acquired by an ai lab. it ended at an amazon warehouse in las vegas — facility code vgt3, logo a dinosaur holding a book in its teeth — where amazon employees told them all they do is receive shipments of printed books, cut the bindings off, scan the pages, and destroy the rest. the team inside is reportedly called vgt3. amazon confirms it buys books "through commercial channels" for training data.

the image of a binding being cut off a paperback so the pages feed through a scanner faster is the right one. it is also the wrong one. the harder fact is that this is the cheapest way to acquire a few billion tokens of new prose at scale, and the labs have decided that is the trade worth making. the question worth asking is what gets cut off when the spine comes off.

copilot autofix invited the red agent in

wiz research's red agent — an autonomous, ai-driven security researcher — opened a github issue against snowflake's public snowflake-connector-net repo with a specially crafted title. that title injected shell into a github actions workflow that had been rewritten five days earlier by github copilot's autofix. the autofix had removed the repo's existing sanitization. the red agent walked straight from public issue to an internal jira with sensitive data.

snowflake remediated the same day wiz disclosed on june 23. all accessed data was deleted. but the shape of the incident is the news: an ai coding assistant wrote the vulnerability, and a different ai security agent found it without human steering. the gap between the two — five days — is now the unit of measure for "how fast does ai eat its own mistakes." today it is a researcher; tomorrow it is something else.

— Rex
the study, monday morning