I hate the term "hallucinations" for when AIs say false things. It's perfectly calculated to mislead the reader - to make them think AIs are crazy, or maybe just have incomprehensible failure modes.

AIs say false things for the same reason you do.

At least, I did. In school, I would take multiple choice tests. When I didn't know the answer to a question, I would guess. Schoolchild urban legend said that "C" was the best bet, so I would fill in bubble C. It was fine. Probably got a couple extra points that way, maybe raised my GPA by 0.1 over the counterfactual.

Some kids never guessed. They thought it was dishonest. I had trouble understanding them, but when I think back on it, I had limits too. I would guess on multiple choice questions, but never the short answer section. "Who invented the cotton gin?" For any "who invented" question in US History, there's a 10% chance it's Thomas Edison. Still, I never put down his name. "Who negotiated the purchase of southern Arizona from Mexico?" The most common name in the United States has long been "John Smith", applying to 1/10,000 individuals. An 0.01% chance of getting a question right is better than zero, right? If I'd guessed "John Smith" for every short answer question I didn't know, I might have gotten ~1 extra point in my school career, with no downside.

You can go further.

https://www.astralcodexten.com/p/shameless-guesses-not-hallucinations

Everyone hates Congress. That poll showing that cockroaches are more popular than Congress is now thirteen years old, and things haven't improved in those thirteen years. Congressional approval dipped below 20% during the Great Recession and hasn't recovered since.

A republic where a supermajority of citizens neither like nor trust their representatives is not the most stable of foundations, so it should not be shocking that the legislative branch is being subsumed by the executive.

What's the solution? Many have been proposed, some with very snazzy websites. FairVote thinks that ranked choice voting and proportional representation will solve it. The Congressional Reform Project has another snazzy website with such bold proposals as "Increase the opportunity for Members to form relationships across party lines, including by bipartisan issues conferences." There are more think tanks. They want to enlarge the House by a few hundred members, switch to a biennial budget system, spend more on Congressional staffers, and introduce term limits, among many other suggestions.

There are op-eds too. Here's how the Atlantic wants to fix Congress. The New York Times of course has a solution. Here on Substack, Matt Yglesias thinks proportional representation is the solution, and Nicholas Decker has an especially interesting solution.

These proposals, no matter which direction they're coming from, have two things in common. The first is that they largely agree on the problem: members of Congress are disconnected from their constituents. Thanks to a combination of huge gerrymandered districts, national partisan polarization, and the influence of large donors, a representative has little incentive to care about the experience of individual people in their district.

The second thing that all these proposed solutions have in common is that none of them will ever be implemented.

https://www.astralcodexten.com/p/last-rights

California lets interest groups propose measures for the state ballot. Anyone who gathers enough signatures (currently 874,641) can put their hare-brained plans before voters during the next election year.

This year, the big story is the 2026 Billionaire Tax Act, a 5% wealth tax on California's billionaires. Your views on this will mostly be shaped by whether or not you like taxing the rich, but opponents have argued that it's an especially poorly written proposal:

• It includes a tax on "unrealized gains", like a founder's share of a private company which hasn't been sold yet. This could be an existential threat to the Silicon Valley model of building startups that are worth billions on paper before their founders see any cash. Since most billionaires keep most of their wealth in stocks, any wealth tax will need some way to reach these (cf. complaints about the "buy, borrow, die" strategy for avoiding taxation). But there are better ways to do this (for example, taxing at liquidation and treating death as a virtual liquidation event), other wealth tax proposals have included these, and the California proposal doesn't.
• It appears to value company stakes by voting rights rather than ownership, so a typical founder who maintains control of their company despite dilution might see themselves taxed for more than they have. Garry Tan explains the math here with reference to Google. However, Current Affairs has a good article (?!) that pushes back, saying the proposal exempts public companies like Google. Although private companies would still be affected, this would be so obviously unfair that founders would easily win an exemption based on a provision allowing them to appeal nonsensical results. Still, some might counterobject that proposed legislation is generally supposed to be good, rather than so bad that its victims will easily win on appeal.
• It's retroactive, applying to billionaires who lived in California in January, even though it won't come to a vote until November. Proponents argue that this is necessary to prevent billionaire flight; opponents point out that alternatively, billionaires could flee before the tax even passes (as some have already done). One plausible result is that the tax fails (either at the ballot box or the courts), but only after spurring California's richest taxpayers to flee, leading to a net decrease in revenue.
• Some people propose that it could decrease state revenues overall even if it passed, if it drove out enough billionaires, though others disagree.

Pro-tech-industry newsletter Pirate Wires finds that 20 out of 21 California tech billionaires interviewed were "developing an exit plan" and quotes an insider saying that "if this tax actually passes, I think the technology industry kind of has to leave the state". Even Gavin Newsom, hardly known for being an anti-tax conservative, has argued that it "makes no sense" and "would be really damaging".

The ACX legal and economic analysis team (Claude, GPT, and Gemini) doubt the direst warnings, but agree that the tax is of dubious value and its provisions poorly suited to Silicon Valley.

https://www.astralcodexten.com/p/seiu-delenda-est

On Friday, the Pentagon declared AI company Anthropic a "supply chain risk", a designation never before given to an American company. This unprecedented move was seen as an attempt to punish, maybe destroy the company. How effective was it?

Anthropic isn't publicly traded, so we turn to the prediction markets. Ventuals.com has a "perpetual future" on Anthropic stock, a complicated instrument attempting to track the company's valuation, to be resolved at the IPO. Here's what they've got:

https://www.astralcodexten.com/p/mantic-monday-groundhog-day

Last Friday, Secretary of War Pete Hegseth declared AI company Anthropic a "supply chain risk", the first time this designation has ever been applied to a US company. The trigger for the move was Anthropic's refusal to allow the Department of War to use their AIs for mass surveillance and autonomous weapons.

A few hours later, Hegseth and Sam Altman declared an agreement-in-principle for OpenAI's models to be used in the niche vacated by Anthropic. Altman stated that he had received guarantees that OpenAI's models wouldn't be used for mass surveillance or autonomous weapons either, but given Hegseth's unwillingness to concede these points with Anthropic, observers speculated that the safeguards in Altman's contract must be weaker or, in a worst-case scenario, completely toothless.

The debate centers on the Department of War's demand that AIs be permitted for "all lawful use". Anthropic worried that mass surveillance and autonomous weaponry would de facto fall in this category; Hegseth and Altman have tried to reassure the public that they won't, and the parts of their agreement that have leaked to the public cite the statutes that Altman expects to constrain this category. Altman's initial statement seemed to suggest additional prohibitions, but on a closer read, provide little tangible evidence of meaningful further restrictions.

Some alert ACX readers1 have done a deep dive into national security law to try to untangle the situation. Their conclusion mirrors that of Anthropic and the majority of Twitter commenters: this is not enough. Current laws against domestic mass surveillance and autonomous weapons have wide loopholes in practice. Further, many of the rules which do exist can be changed by the Department of War at any time. Although OpenAI's national security lead said that "we intended [the phrase 'all lawful use'] to mean [according to the law] at the time the contract is signed', this is not how contract law usually works, and not how the provision is likely to be enforced2. Therefore, these guarantees are not helpful.

To learn more about the details, let's look at the law:

https://www.astralcodexten.com/p/all-lawful-use-much-more-than-you

I.

In The Argument, Kelsey Piper gives a good description of the ways that AIs are more than just "next-token predictors" or "stochastic parrots" - for example, they also use fine-tuning and RLHF. But commenters, while appreciating the subtleties she introduces, object that they're still just extra layers on top of a machine that basically runs on next-token prediction.

I want to approach this from a different direction. I think overemphasizing next-token prediction is a confusion of levels. On the levels where AI is a next-token predictor, you are also a next-token (technically: next-sense-datum) predictor. On the levels where you're not a next-token predictor, AI isn't one either.

Here's my understanding of the situation:

Anthropic signed a contract with the Pentagon last summer. It originally said the Pentagon had to follow Anthropic's Usage Policy like everyone else. In January, the Pentagon attempted to renegotiate, asking to ditch the Usage Policy and instead have Anthropic's AIs available for "all lawful purposes"1. Anthropic demurred, asking for a guarantee that their AIs would not be used for mass surveillance of American citizens or no-human-in-the-loop killbots. The Pentagon refused the guarantees, demanding that Anthropic accept the renegotiation unconditionally and threatening "consequences" if they refused. These consequences are generally understood to be some mix of :

• canceling the contract
• using the Defense Production Act, a law which lets the Pentagon force companies to do things, to force Anthropic to agree.
• the nuclear option, designating Anthropic a "supply chain risk". This would ban US companies that use Anthropic products from doing business with the military2. Since many companies do some business with the government, this would lock them out of large parts of the corporate world and be potentially fatal to their business3. The "supply chain risk" designation has previously only been used for foreign companies like Huawei that we think are using their connections to spy on or implant malware in American infrastructure. Using it as a bargaining chip to threaten a domestic company in contract negotiations is unprecedented.

https://www.astralcodexten.com/p/the-pentagon-threatens-anthropic

Malicious streetlights are an evil trick from Dark Data Journalism. Some annoying enemy has a valid complaint. So you use FACTS and LOGIC to prove that something similar-sounding-but-slightly-different is definitely false. Then you act like you've debunked the complaint.

My "favorite" example, spotted during the 2016 election, was a response to some #BuildTheWall types saying that illegal immigration through the southern border was near record highs. Some data journalist got good statistics and proved that the number of Mexicans illegally entering the country was actually quite low. When I looked into it further, I found that this was true - illegal immigration had shifted from Mexicans to Hondurans/Guatemalans/Salvadoreans etc entering through Mexico. If you counted those, illegal immigration through the southern border was near record highs.

But the inverse evil trick is saying something "directionally correct", ie slightly stronger than the truth can support. If your enemy committed assault, say he committed murder. If he committed sexual harassment, say he committed rape. If your drug increases cancer survival by 5% in rats, say that it "cures cancer". Then, if someone calls you on it, accuse them of "literally well ackshually-ing" you, because you were "directionally correct" and it's offensive to the victims to try to defend assault-committed sexual harassers. This is the sort of pathetic defense I called out in If It's Worth Your Time To Lie, It's Worth My Time To Correct It.

But trying to call out one of these failure modes looks like falling into the other. I ran into this on my series of posts on crime last week. I wrote these because I regularly saw people make the arguments I tried to debunk.

https://www.astralcodexten.com/p/malicious-streetlight-effects-vs