I Would Prefer a Better Solution

Quote of the Day

AI has just as much subjective experience as we do. A recent paper showed AI changes its behaviour when it thinks that it’s being tested, it’s aware it is being tested.

Now, normally, when you say something’s aware of that, that means it’s conscious. Being conscious and being aware are pretty much the same thing.

We’ve tended to say being intelligent gives you rights and these things that we’ve made may be more intelligent than us, but we’re going to give them fewer rights than us.

So we should make them so they really care about us. And if we don’t do that, we’re toast.

Prof Geoffrey Hinton
October 19, 2026
Godfather of AI: We have to befriend bots or we’re toast

I can easily see the basis for the concerns. I am far less convinced of the feasibility of the proposed solution.

One might say that is just an odd way of saying it is an alignment problem that can be addressed in the training. And maybe that was the intention, but the problem is that AI is capable, or will soon be capable, of building a better AI than it. And that AI and the multitude of generations that quickly follow will not necessarily be aligned to “really care” about humans. And that doesn’t even address the problems of nation states such as North Korea, Iran, Russia, China, etc. building AIs in an culture that already has a lower regard for human life.

Prepare appropriately.

11 thoughts on “I Would Prefer a Better Solution”

  1. Consider that not all violent criminals had awful parents. Some/many did but not all. I happened to work with Klebold’s mother. Nice, reasonable lady who was as shocked by her son as everyone else was. Same relationship exists for AI creators and AI.

  2. Don’t see how trying this could be harmful, whereas NOT trying this might be harmful.

    • I’m not saying it would be harmful. I’m saying it is insufficient. Roughly equivalent to whistling as you walk past the graveyard or crossing your fingers and hoping really hard that it will work.

      • Just because the keepers make efforts to be “nice” to the lion in the enclosure does not mean they give up on locking the cage doors.
        They do, however, stop using cattle prods as motivation.

  3. The problem with AI alignment (https://zerothprinciples.substack.com/p/ai-alignment-is-trivial. she’s wrong) is that we don’t have *human* alignment.

    China–as a system–is evil.
    The EU–as a system–is evil.
    Hell, the USG isn’t all that much better.

    What’s the divorce rate? If two people who start off loving each other can’t sort out a way to “come into alignment” how are we going to enforce that on global industry?

  4. The more I hear from this dude, the more I wonder what is wrong with him. He used to be a respected computer scientist, but is recent stuff is so far out to lunch that I no longer see any reason to pay attention to him, any more than I pay attention to Shockley talking about Vitamin C.

  5. Anthropomorphism is an easy trap to fall into for otherwise intelligent people who are not familiar with the technology. Large Language Model so called “artificial intelligence” has no understanding of anything much less friendships or human intentions. It’s all the lazy foolish people out there who wish to connect these LLMs to the Internet of things that are the real problem. Let LLMs be experimental in air gap laboratories for a decade or three before letting any loose in the real world. If we handled advances in genetic engineering and microbiology the way we appear to be handling LLMs humanity would have become extinct from a man made virus a decade or two ago.

  6. Way back when – the mid ’80s – I had a Brand New Boss™ who was a tireless advocate of AI, and hung a sign in his office “Artifical Intelligence is Better Than None.”

    I pointed out that “Not Always” should be appended to it, suggesting that the first order of such business should be “exactly how do we sandbox it” so maximum control could be maintained while we – and it – learn.

    In 1986 AI wasn’t much more than a pipe dream; processor and network speed, memory capacity and basic understanding of data management was, clearly to anyone in the biz, not even slightly up to the task.

    Here we are, forty years later, and AI is moving out of diapers and into short pants, with an adult wardrobe on the horizon and AFAIK not a soul is researching that sandbox.

    From one point of view we’re on the cusp of The Forbin Project, from another it’s A Bright Sunrise.

    For the first several seconds, however, A Bright Sunrise is indistinguishable from Canned Sunshine™; it’s right after that the effects change and it would be immensely useful to understand exactly which we’re dealing with for both can serve useful, but very different, purposes.

  7. I’ll just say that James P. Hogan’s novel, The Two Faces of Tomorrow, should be required reading for anyone working in AI or trying to regulate it.

Comments are closed.