I have now read so many “ChatGPT can do X job better than workers” papers, and I don’t think that I’ve ever found one that wasn’t at least flawed if not complete bunk once I pulled up the actual paper. I wrote about this a year ago, and I’ve since done the occasional follow-up on specific articles, including an official response to one of the most dishonest published papers that I’ve ever read that just itself passed peer review and is awaiting publication.

That academics are still “bench-marking” ChatGPT like this, a full year after I wrote that, is genuinely astounding to me on so many levels. I don’t even have anything left to say about it at this point. At least fewer of them are now purposefully designing their experiments to conclude that AI is awesome, and are coming to the obvious conclusion that ChatGPT cannot actually replace doctors, because of course it can’t.

This is my favorite one of these ChatGPT-as-doctor studies to date. It concluded that “GPT-4 ranked higher than the majority of physicians” on their exams. In reality, it actually can’t do the exam, so the researchers made a special, ChatGPT-friendly version of the exam for the sole purpose of concluding that ChatGPT is better than humans.

Because GPT models cannot interpret images, questions including imaging analysis, such as those related to ultrasound, electrocardiography, x-ray, magnetic resonance, computed tomography, and positron emission tomography/computed tomography imaging, were excluded.

Just a bunch of serious doctors at serious hospitals showing their whole ass.

Unfortunately, COP28 is taking it seriously, seen here with “both sides” reporting from the NYT.

Not directly to your question, but I dislike this NPR article very much.

Mwandjalulu dreamed of becoming a carpenter or electrician as a child. And now he’s fulfilling that dream. But that also makes him an exception to the rule. While Gen Z — often described as people born between 1997 and 2012 — is on track to become the most educated generation, fewer young folks are opting for traditionally hands-on jobs in the skilled trade and technical industries.

The entire article contains a buried classist assumption. Carpenters have just as much a reason to study theater, literature, or philosophy as, say, project managers at tech companies (those three examples are from PMs that I’ve worked with). Being educated and a carpenter are only in tension because of decisions that we’ve made, because having read Plato has as much in common with being a carpenter as it does with being a PM. Conversely, it would be fucking lit if our society had the most educated plumbers and carpenters in the world.

NPR here is treating school as job training, which is, in my opinion, the root problem. Job training is definitely a part of school, but school and society writ large have a much deeper relationship: An educated public is necessary for a functioning democracy. 1 in 5 Americans is illiterate. If we want a functioning democracy, then we need to invest in everyone’s education for its own sake, rather than treat it as a distinguishing feature between lower classes and upper ones, and we need to treat blue collar workers as people who also might wish to be intellectually fulfilled, rather than as a monolithic class of people who have some innate desire to work with their hands and avoid book learning (though those kinds of people need also be welcomed).

Occupations such as auto technician with aging workforces have the U.S. Chamber of Commerce warning of a “massive” shortage of skilled workers in 2023.

This is your regular reminder that the Chamber of Commerce is a private entity that represents capital. Everything that they say should be taken with a grain of salt. There’s a massive shortage of skilled workers for the rates that businesses are willing to pay, which has been stagnant for decades as corporate profits have gone up. If you open literally any business and offer candidates enough money, you’ll have a line out the door to apply.

Very well could be. At this point, I’m so suspicious of all these reports. It feels like trying to figure out what’s happening inside a company while relying only on their ads and PR communications: The only thing that I do know for sure is that everyone involved wants more money and is full of shit.

US Leads World in Credulous Reports of ‘Lagging Behind’ Russia. The American military, its allies, and the various think-tanks it funds, either directly or indirectly, generate these reports to justify forever increasing the military budget.

To be clear, I wasn’t advocating for organized violence as a good tactic. I was just picking a simple example.

I still think that Bevins’s history and analysis has merit, even if you disagree with his conclusions. I’ve read at least two books by anarchists that put forth similar concepts of legibility: Graeber’s “Utopia of Rules” and James Scott’s “Seeing like a State” (which I actually read to write this post and have a bajillion opinions about, but that’s a post for another day). Regardless of your stance on whether your movement should or shouldn’t be legible, you have to understand legibility, both to the state, and to other capitalist powers like, say, social media (to pick one at random 😉 ).

I once again disagree with your characterization of the book.

You realize how funny it is that you post this in an Anarchist community?

That’s stupid. Anarchist revolutionary theory and historical practice are full of ideas that are perfectly compatible with this analysis, even if Bevins himself is clearly not an anarchist. There is no more legible act to the state than organized violence, for example.

I’m not sure why you’ve taken this unpleasant posture towards me. I’m genuinely here for a discussion, but this is my last response if you keep acting like I’m some sort of uncultured idiot that needs you “to start from the basics 😒”

Yeah, again, I take pretty strong issue with your characterization of Bevins’s stance. Have you actually read the book? I think that this is an interesting and worthwhile discussion, but I also don’t want to go in circles if you haven’t…

When he says that they’re illegible to state power, he doesn’t mean that they want to appeal to the people currently in power (and maybe this is a conflation that I accidentally invite in my own write-up). He means that they cannot participate in state power as an institutional apparatus, be it as reformists or revolutionaries.

I get what you’re saying, and I agree with a lot of it (but not all of it), but you’re just not responding to an argument that Bevins makes, at least in how I read him. You are responding to one that many in western media did in fact make, and I agree with you in that context, but that was just not my reading of Bevins at all.

I don’t think its wired to critique someone for having a widely different interpretation of what happened than multiple others that were directly involved and then taking this very peculiar subjective interpretation to make wide sweeping (and IMHO wrong) conclusions about what we should learn from it.

It is because that’s literally what the book is about. The book is addressing that very phenomenon as its core thesis. That’s exactly what he is talking about when he says that the protests are illegible. If someone says “people disagree a lot about what happened and that’s a problem” responding to that by saying “i disagree about what happened” isn’t really engaging with the argument.

My impression is that Bevin started out with a preconsived notion and then kinda made up a retrospective narrative of these protests to fit to that.

I’m sorry but I don’t think that anyone who has actually read the book in good faith can come to that conclusion.

edit: added more explanation

That’s kind of a weird critique, because it’s actually consistent with the book. He spends a lot of time talking about how wildly different every person’s interpretation of the event is, and that’s kind of the problem. It’s part of why these movements are illegible to power. He’s very clear that this is his interpretation, based on his own contacts, experience, and extensive research, but that it’s not going to be the same as everyone else’s.

Same is true with the moniker. Whether or not the people on the ground felt that way about it or not, that story, fabricated without input from those on the ground, is what ended up creating meaning out of the movement, at least insomuch as power is concerned. That’s like the core thesis of the book: The problem with that wave of protests was not being able to assert their own meaning over their actions. The meaning was created for them by people like western media, and they weren’t able to organize their own narrative, choose their own representatives, etc.

edit to add: IIRC, he even specifically discusses how the different people in the core group of Brazilian organizers disagree on what happened.

Oh hey I wrote that lol.

Not all protests for Gaza were meant to gain engagement, many were organized to cause direct economic disruption to those that profit from the war, that is a goal.

I actually totally agree with you. I should’ve been more careful in the text to distinguish between those two very different kinds of actions. I actually really, really like things that disrupt those that profit, but those are not nearly as common as going to the local park or whatever. I might throw in a footnote to clarify.

All these always do the same thing.

Researchers reduced [the task] to producing a plausible corpus of text, and then published the not-so-shocking results that the thing that is good at generating plausible text did a good job generating plausible text.

From the OP , buried deep in the methodology :

Because GPT models cannot interpret images, questions including imaging analysis, such as those related to ultrasound, electrocardiography, x-ray, magnetic resonance, computed tomography, and positron emission tomography/computed tomography imaging, were excluded.

Yet here’s their conclusion :

The advancement from GPT-3.5 to GPT-4 marks a critical milestone in which LLMs achieved physician-level performance. These findings underscore the potential maturity of LLM technology, urging the medical community to explore its widespread applications.

It’s literally always the same. They reduce a task such that chatgpt can do it then report that it can do to in the headline, with the caveats buried way later in the text.

We need to set aside our petty differences and fight the true enemy: bloated IDEs.

AI systems in the future, since it helps us understand how difficult they might be to deal with," lead author Evan Hubinger, an artificial general intelligence safety research scientist at Anthropic, an AI research company, told Live Science in an email.

The media needs to stop falling for this. This is a “pre-print,” aka a non-peer-reviewed paper, published by the AI company itself. These companies are quickly learning that, with the AI hype, they can get free marketing by pretending to do “research” on their own product. It doesn’t matter what the conclusion is, whether it’s very cool and going to save us or very scary and we should all be afraid, so long as its attention grabbing.

If the media wants to report on it, fine, but don’t legitimize it by pretending that it’s “researchers” when it’s the company itself. The point of journalism is to speak truth to power, not regurgitate what the powerful say.

Yeah, as always, the devil is in the details. For now I think that we need a simple and clear articulation of the main idea. In the exceedingly unlikely event that it ever gets traction, I look forward to hammering out the many nuances.

It’s not a solution, but as a mitigation, I’m trying to push the idea of an internet right of way into the public consciousness. Here’s the thesis statement from my write-up:

I propose that if a company wants to grow by allowing open access to its services to the public, then that access should create a legal right of way. Any features that were open to users cannot then be closed off so long as the company remains operational. We need an Internet Rights of Way Act, which enforces digital footpaths. Companies shouldn’t be allowed to create little paths into their sites, only to delete them, forcing guests to pay if they wish to maintain access to the networks that they built, the posts that they wrote, or whatever else it is that they were doing there.

As I explain in the link, rights of way already exist for the physical world, so it’s easily explained to even the less technically inclined, and give us a useful legal framework for how they should work.

I don’t really agree with this. It is the answer that I think classical economics would give but I just don’t think it’s useful. For one, it ignores politics. Large corporations also have bought our government, and a few large wealth management funds like vanguard own a de facto controlling share in many public companies, oftentimes including virtually an entire industry, such that competition between them isn’t really incentived as much as financial shenanigans and other Jack Welch style shit.

Some scholars (i think I read this in Adrienne bullers value of a whale, which is basically basis for this entire comment) even argue that we’ve reached a point where it might be more useful to think of our economy as a planned economy, but planned by finance instead of a state central authority.

All that is to say: why would we expect competition to grow, as you suggest, when the current companies already won, and therefore have the power to crush competition? They’ve already dismantled so many of the antimonopoly and other regulations standing in their way. The classical economics argument treats these new better companies as just sorta rising out of the aether but in reality there’s a whole political context that is probably worth considering.

From Graeber’s The Dawn of Everything:

For instance, if Pinker is correct, then any sane person who had to choose between (a) the violent chaos and abject poverty of the ‘tribal’ stage in human development and (b) the relative security and prosperity of Western civilization would not hesitate to leap for safety. But empirical data is available here, and it suggests something is very wrong with Pinker’s conclusions.

Over the last several centuries, there have been numerous occasions when individuals found themselves in a position to make precisely this choice – and they almost never go the way Pinker would have predicted. Some have left us clear, rational explanations for why they made the choices they did.

Graeber goes on to give a couple of these accounts. They tend to mention a loneliness associated with “western civilization,” as well as a feeling that I think lines up very well with what Marx described as alienation.

Some emphasized the virtues of freedom they found in Native American societies, including sexual freedom, but also freedom from the expectation of constant toil in pursuit of land and wealth.

Later in the book, and I apologize that I can’t find the reference right now, he comes back to this topic for a little bit, and talks about the depths of relationships that these people describe, and how their relationships in the “civilized” world are more shallow and less satisfying. Deep human relationships are the opposite of fake, so I think here we have a point in favor of “yes.”

Add to that that the concept of “privacy” as we know it is relatively new. It’s been 10+ years since I read a book about this, the title of which I can’t even remember, but it argued that the expectation of domestic privacy, even from one’s own family, is a phenomenon from the last few hundred years, especially outside the elite. People lived far, far more communally, with the expectation that they just were in each other’s business more. I’d argue that it’s a lot harder to be fake if you can’t hide who you really are.

Between those two things, I think it’s reasonable to argue that yes, society has gotten more fake.

If you’re a US citizen, consider joining the DSA. I’m a long-time member. I think it’s safe to say the DSA has, historically, been a disorganized parody of a leftist organization (as much as I love and respect many of the people in it that I’ve worked with), but things are changing. There’s an effort with momentum to turn it into a functioning political party, and not a bullshit green party style party which runs a candidate every four years while being functionally indistinguishable from a grift, but to put in the real work from the ground up to make a party that cares about winning elections and materially making our lives better.

The time to do this was 20 years ago, but we can’t keep delaying it. It’s now fully unconscionable to throw up our hands after some halfhearted discussions about FPTP and game theory every four years while actively watching our world deteriorate. There are other ways, but they don’t start at the ballot box, and they all involve organizing. This is true even if your politics and mine are different. If you care about our death machine funding a genocide, get involved with something, even if it’s the democratic party. It’s fucking boring. It feels like a larp. It’s a tedious ways to spend your Thursday nights after work. I get all that, but we need people who care about human life involved, even if our politics aren’t perfectly aligned, because that’s how you make broad, functioning, powerful coalitions that get shit done.

Not if you’re too busy between your two jobs manually training the LLM models and supervising the supposedly autonomous cars to make rent.