From TPM Reader FP …

I’ve worked for decades on language models, as a researcher in academia and industry, and as a research manager whose teams have brought language models into products several years before the current excitement. I’m enjoying your commentaries on the topic, so I’m writing with a bit of historical perspective and connections that might be helpful.

The connection between language models and games goes back a long way, arguably to Turing and Shannon. Consider a reader who is shown the words (or letters, the difference is not significant) one by one, from the books in a library, and has to bet on the next still unseen word. If the reader is sufficiently educated in English language and culture, they have an almost sure bet in continuing “… to be or not … ” However, if the text was “… classical concert goers prefer Beethoven to … ” there are several possible continuations, but there’s still some predictability: the following word is more likely to be “Stockhausen” than to be “Cheetos”: text from the library tends to have some thematic coherence, in this case musical preferences, rather than mixing music and snacks. 

A friend sent me an article from back in March 2016 which provides some interesting perspective on the current resurgence of Social Security politics and the various Republicans vying to be the “post-Trump” candidate for President while Trump refuses to leave the stage. It also has particular relevance to Ron DeSantis, which we’ll get to in a moment.

But first some context.

The piece is a Times article from March 2016. So it is early in the Trump takeover of the GOP but when it still wasn’t entirely clear he’d be able to pull it off. The subtext of the article is that while many Republicans focused either on the power of Trump’s chaotic personality or the red meat of immigrant bans and xenophobia to explain his success, there was something else in the mix. There was a whole population of people who had closed the door on ever supporting Democrats but were left entirely cold by the GOP’s reflexive focus on tax cuts, free trade and cutting “entitlements.”

I’ve mentioned this many times before. But it’s one of the true privileges of this job to have the asset of this site’s readership as I explore new issues raised in the news. Readers who were only readers for years and now sometimes decades become active participants as the site’s focus shifts to their area of expertise. “20 year reader here,” said one last night, “finally you address a topic I’m an expert in!”

I’ve heard from a range of readers who are top executives and engineers at companies on the forefront of artificial intelligence, computer science academics, people who have some angle of expertise on the topic. I’ve been hearing from more people on the “pro-AI” side of things. But “pro” or “con” doesn’t really do justice to the conversation.

I just wanted to drop this in here because it’s an important set of facts that I suspect will be a big issue both in the coming GOP primary and — if DeSantis gets the GOP nomination — during the 2024 general election.

A week ago I noted that, contrary to press claims that only a few GOP outliers have called for cutting or phasing out Social Security, the great majority of House Republicans had already endorsed doing so during this fiscal year. There’s a lot of press special pleading surrounding proposals for Social Security cuts: It’s only Rick Scott or Ron Johnson or Mike Lee. In fact, such proposals are GOP orthodoxy and have been for decades. The Republican Study Committee, which is a caucus including about three-quarters of House Republicans, endorsed a proposed budget for this fiscal year calling for Social Security privatization, benefit cuts (through changed COLA formulas), raising the retirement age to 70 and more.

Amidst all the storm and drama I’ve been enjoying learning more about just how the current crop of AI applications (LLMs, or Large Language Models) operate. I don’t want to imply that I understand them at a deep level — my programming career peaked with some piecework programming in Visual Basic that I did in grad school. (Actually, there was also that extremely elegant bubble sort program I wrote in middle school. But I digress …) But it has at least given me some sense of the roots of the limitations that are coming to the fore. Specifically — but not only — the way applications like ChatGPT frequently fabricate false claims clothed in the trappings of facticity. I’m not just claiming this new protein structure exists. I’m citing the specific journal article with the page numbers and date and authors, etc. It’s got to be true!

To the great distress of many MAGA Republicans, President Biden made a surprise visit to Kyiv early this morning (U.S. time). Here are some of the most striking images.

I’ve had questions from a number of you about why there’s been so little mainstream media pick up of Seymour Hersh’s claim that the U.S. destroyed the Nord Stream pipeline last year. As a factual matter, I think it’s highly unlikely that this is true. But my point here isn’t to dispute the report. People know that Hersh is one of the most high profile and celebrated investigative reporters of the last half century. So how is it he’s suddenly seen as an unreliable source of information?

On this point I think I can shed some light.

The White House has congratulated Rick Scott for confessing his desire to destroy Social Security and seeking forgiveness.

On Wednesday we did a TPM Inside Briefing with economist and Times columnist Paul Krugman. I already shared with you the portion of the discussion in which we talked about Democrats’ options for extraordinary/heroic measures to avert a U.S. credit default if House Republicans force the issue. But we covered a lot of other important ground directly relevant to the biggest news stories of the day — the financial viability of Social Security (and Medicare) and the sustainability of the aggregate federal debt in particular.

A substantial percentage of the U.S. federal debt (about a quarter of it) has been run up in just the last three years — largely in response to the COVID pandemic crisis. Is the scale of the federal debt sustainable? Does it require drastic action? Krugman’s answer was basically, no. There’s no crisis. Despite what you hear, it’s not unsustainable at all.

On Social Security, same thing. At some point in the future — probably about a dozen years from now — Social Security will need more revenue to cover all the benefits of the people currently in retirement under Social Security. At that point, we’ll have a simple decision on whether to raise taxes to cover those benefits or cut the benefits. It’s not really an economics decision. Both are totally doable in macro-economic terms. It’s really a values and politics question. Which is more important? Preserving the benefits of retirement beneficiaries (and others covered by Social Security) or preventing higher taxes on upper-income earners? You don’t need to be a PhD economist to decide this question.

Critically, does the problem get worse the longer we put off these decisions? Again, no. All the claims to the contrary are a kind of bums’ rush to force one of the answers over the other — cuts as opposed to taxes.

All these questions and answers about debt, Social Security, Medicare, and the debt ceiling were highly clarifying for me. And I suspect they will be for you as well. If you’re a member, join me after the jump for the full TPM Inside Briefing with Paul Krugman.

I had heard a bit about this in discussions of inaccurate information from AI engines like ChatGPT. But hearing it directly from TPM Readers AT (a research librarian) and JC brought it into sharper relief for me. Inaccurate information can mean a lot of things. You can say Thomas Jefferson died in 1846 when in fact he died in 1826. Other inaccuracies are more complex and pernicious.