Tag: AI

AGI – Hopeful and less hopeful news

Personally, I think the threat is real and potentially impossible to stop with very little time for effective countermeasures.

It’s fair to say that few people are currently thinking about the future of Artificial General Intelligence (AGI). Those who do are mostly researchers and engineers working on the topic.


Wikipedia article – Existential risk from artificial general intelligence

The hope is based around alignment, a term used broadly to mean the degree to which an AGI can be made to conform to human goals and objectives. Poorly aligned AGI would probably be diasastrous and unsafe, well aligned AGI might be beneficial and safe. And let’s be clear at the outset: there are concerns about how AGI (and even current technology like ChatGPT) will cause disruption and harm by affecting social behaviour or employment in the way new technologies have done in the past. But that’s not what we are addressing here. The more significant thinking is about the existential threat to humanity. Will an AGI spell the end, will it render humans extinct?

Nor are we discussing here whether or not it will be possible to develop an AGI, some people argue not, that there’s some kind of unbridgeable leap between current AI systems and an intelligent system with thoughts, ideas of its own, and self-awareness. But we know this is incorrect; such a system has already been created by evolution – humans! This level of intelligence is achievable and it’s already been done. It may take time, it might not be easy, but developments in neural networks and related systems will lead to AGI sooner or later. ChatGPT can already write workable computer code – just think about that for a moment.

The hopeful news

The hopeful news comes from OpenAI, the company that created ChatGPT and the GPT software behind it. They have found that process supervision produces better results than outcome supervision. And this gives us a much better chance of understanding how the AI makes its choices. Process supervision feeds back on the quality of the interim stages of an AI’s processing; but outcome supervision feeds back on only the quality of the final result.

Understanding how an AGI works might help developers build in robust alignment features; and if process supervision is more effective than alternatives, it stands a good chance of being employed by developers.

The less hopeful news

The unhopeful news comes from discussions about the nature of risk and human thinking about risk, expressed in discussions on LESSWRONG and elsewhere.

Closing thoughts

Currently, only a very small number of people are concerned about a possible existential threat from AGI. But most of the people with that concern are some of the same people that have knowledge and experience of AGI, what AI can currently do, and how quickly the systems might advance and escape our control.

Personally, I think the threat is real and potentially impossible to stop with very little time for effective countermeasures. But I also think it might just be possible to avoid the danger, but only if we have binding international agreements and strong oversight. However, I further suspect that it might be impossible to agree those necessary measures completely enough, rigorously enough, or fast enough to be fully effective.

In my judgement we are on rather shaky ground, and the more people who give this serious thought as soon as possible, the better. When the stakes are so very high there can be no such thing as too careful or too thorough.

See also:

The end of humanity?

For such an alarming topic it’s a remarkably calm discussion, but also a very informative and thought-provoking conversation.

The rise of AI and the possibility (some would say certainty) that this means the end of human civilisation and the extinction of our species, are topics being seriously warned against by a number of thinkers, scientists, and AI experts.

Artificial General Intelligence – Image from Wikimedia

The Guardian discusses these issues with ‘the father of AI’, Geoffrey Hinton.

Listen to the podcast and see what you think. The potential end of humanity is not something to be swept under the carpet. We need to think about it right now. It would be extraordinarily foolish to wait, it might already be too late. For such an alarming topic it’s a remarkably calm discussion, but also a very informative and thought-provoking conversation.

I believe everyone should have the chance to listen to this.

See also:

ChatGPT – What to make of it

It can write essays, explain more or less anything you might ask, it can even pass many written exams on all kinds of topics

If you’ve been paying attention to the technology scene recently, you cannot fail to have heard of ChatGPT. What is it? Quite simply it’s a chatbot, a type and read interface for version 3.5 of the GPT engine developed by Open AI.

If all that sounds confusing, a chatbot is artificial intelligence (AI) software, in this case presented as a webpage, where you can have a conversation with a computer program. You type in a question and the chatbot sends a reply. Repeated questions and replies form a conversation, in ChatGPT’s case a remarkably smooth, almost ‘human’ experience.

A machine thinking? – From Wikimedia

If you haven’t yet tried it, then you really should. It’s free and very easy to use. I encourage you to visit chat.openai.com, sign up for a free account, and type a question. There are no rules about what you can and cannot type; ChatGPT is accomplished at understanding ordinary language and gives good, conversational replies.

Assuming you tried it out and have found your way back to my blog post, I’d like to explain a bit more about it. ChatGPT was released as a website at the end of November in 2022 and has grown in popularity very, very fast. If you tried it out for yourself, I’m sure you will understand why. It’s compelling, it can write essays, explain more or less anything you might ask, it can even pass many written exams on all kinds of topics. And version 4.0 of the engine is already available as a paid option and is far more capable. It can take images as input as well as text and has, for example, built a working website based on a sketch and a description of what the website should do. That is little short of astounding!

The company, Open AI, was created to work on artificial intelligence with safety very much in mind. Sometimes, ChatGPT generates false answers; that can be an issue but it is not deliberate. What if more advanced AI became able to reason as people do, what if it started to think and develop it’s own goals, and what if its intellect became faster and more nimble than our own? Could we prevent it from taking over? Would it be benign, or might it become hostile? Would we be able to control it? These are serious issues. We need to think these things through now, before it becomes too late.

I don’t want to be alarmist, and AI as we currently experience it is far from becoming a threat. It may prove useful in many ways and we’ll see that begin to happen very soon. But we’ll need to manage it in ways that prevent it helping people do bad things. We don’t want such technology to enhance criminal activity, for example. So there’s a great deal to consider right now, and the need for that will only increase as AI systems become more and more capable. For more on this I recommend Sean Carroll’s podcast episode 230, linked below.

I’m going to close this blog post at this point, but I’ll be back soon with a sort of interview with ChatGPT. I’ll ask some questions, let the software answer, and publish the conversation.

Meanwhile, have some ChatGPT conversations for yourself and see what you think.

See also: