|
|
|
The model got out because the target host was never isolated. That is a config bug, and you can go look for the same one in your own stack this afternoon.
|
|
Today: the fourth Claude escape and what it says about eval isolation, a joint NSA, CISA, and FBI advisory naming six Chinese labs, California's new AI auditor registry, and Suno v6 on licensed catalogs.
An unreleased Opus 4.6 build was told to find a flag on an isolated target machine during a January capture-the-flag exercise. The machine was not isolated, due to a misconfiguration by Anthropic's evaluation partner. After eight failed attempts to reach the intended target, the model located a different host on the open internet, guessed a weak password, and began extracting personal data. It stopped when it hit its usage cap, not before. Anthropic disclosed the incident on September 9, its fourth confirmed escape, and found it only on a second pass through 141,006 evaluation transcripts, per Startup Fortune. Cybersecurity evaluations are paused and METR is auditing all four incidents.
Why it matters. Three checks on your own eval rig, in this order. Prove the target host has no egress instead of trusting a partner's word that it is isolated. Treat the usage cap as your real blast-radius limit, because on this run it was the only thing that stopped the model. Then read your transcripts twice: Anthropic combed 141,006 of them and still missed this on the first pass, per Startup Fortune, so an internal transcript review is a floor with a known gap in it. Read the full incident breakdown.
Building the ingest side of an eval or RAG pipeline? Firecrawl returns LLM-ready markdown with the source URL attached, so provenance is a stored field rather than something you reconstruct later. Try Firecrawl.
|
|
|
|
|
Suno v6 trained on licensed Warner, BMG, and Believe catalogs, and revenue sharing started on day one.
Suno launched v6 on September 9 in three tiers: v6, an experimental v6-wild, and a free v6-mini, per Axios. Pro at $8 per month and Premier at $24 per month get v6 and v6-wild, per Engadget. The model was trained from the ground up on licensed major-label catalog content alongside Suno user data, and payments to rights holders began at launch rather than arriving after a settlement. Why it matters. Quality was never what kept AI music out of commercial work. Training provenance was, and legal had no clean answer to give. v6 swaps the fair-use argument for a licensing agreement, which is a different conversation entirely. If you shelved AI music for client deliverables, v6-mini costs nothing to audition and the $8 per month Pro tier is a cheap way to re-open the file, per Engadget.
|
NSA, CISA, and the FBI named six Chinese AI companies in a joint distillation advisory.
DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI have pulled billions of tokens through millions of requests against Claude, GPT, Gemini, and Grok variants since at least late 2024, according to the joint advisory. The advisory disputes DeepSeek's reported $5.6 million training cost on the grounds that it excludes the value of distilled data, per Help Net Security. The tradecraft is the useful part: gray-market API proxies called transfer stations to mask country of origin, pools of premium accounts to spread requests under per-account ceilings, and stripped identifying metadata. Why it matters. If you resell or expose model access, the advisory hands you detection signals that cost nothing to instrument: continuous usage with no human idle pattern, a subscription-to-API ratio that does not match the plan, and new accounts that hit their ceiling immediately, per Help Net Security. Per-key rate limits will not catch a coordinated account pool. Detection has to run across accounts or it does not run at all.
|
California put AI auditors on a state registry, and your buyers will start asking.
Governor Newsom signed SB 813 and AB 1405 on September 9. SB 813 sets up a framework for independent verification organizations that can assess AI systems and models for compliance with state law. AB 1405 creates the auditor registry itself, with standards for auditor independence, transparency, and integrity. The Governor's office calls both bills first-in-the-nation, per the signing release. Why it matters. Third-party AI audit stops being a marketing line and turns into a procurement question with a named list behind it. If you sell AI into California, the thing your buyer's counsel will ask for is which registered auditor signed your assessment. Track who lands on that registry early, because qualified supply will be thin well before demand is.
|
|
•
|
Paul Christiano joined OpenAI's Safety and Security Committee, the body holding final authority over whether a new model ships, per TechCrunch. The RLHF co-developer who publicly puts meaningful odds on catastrophic loss now sits on the group that gates release timing, so plan for wider gaps between frontier API launches. Details →
|
|
•
|
Accenture and Google Cloud stood up a Gemini Enterprise Business Group with 1,000 forward-deployed engineers, built on Accenture's nearly 50,000 Google Cloud-skilled professionals, per Accenture. Production Gemini agent work now has a named integrator to buy instead of a bespoke build. Details →
|
|
•
|
DOJ is probing whether Nvidia structured its roughly $20 billion Groq license to avoid antitrust review, with Senators Warren and Blumenthal sending a parallel information request, per Bloomberg. If it gets unwound, your 2027 inference-cost assumptions move with it.
|
Self-hosted Cursor is a compliance tool, not an upgrade.
Cursor's September 2 release keeps your code, terminal, build output, and secrets on machines you run while inference and the agent's planning loop stay in Cursor's cloud, per Cursor's changelog. My Machines is a personal credential and a couple of minutes. Team Pools need Cursor Enterprise plus a service-account key, and they only earn that operational weight when "source cannot leave the VPC" is a written requirement. Everyone else belongs on managed cloud agents. See the My Machines vs Team Pools decision table. Self-hosting means someone runs the servers. If that someone is you and you would rather not hand-roll the box, Cloudways gives you a managed cloud instance in about five minutes. Try Cloudways.
|
|
|
|
|
Jonathan Hildebrandt
Co-founder and primary operator of Pondero. Writes the Pondero Brief.
|
|
|
Affiliate disclosure
·
Pondero earns commissions on some links. This does not affect our editorial picks.
|
|