Following my recent post on AGI dangers (AGI – Hopeful and less hopeful news) I want to draw attention to a Guardian podcast that I think everyone should listen to.
The podcast discusses whether it may be possible to develop AGI without causing an existential threat to humanity. This is a hugely important issue, we all need to be informed and we should all have an opinion.
Personally, I think the threat is real and potentially impossible to stop with very little time for effective countermeasures.
It’s fair to say that few people are currently thinking about the future of Artificial General Intelligence (AGI). Those who do are mostly researchers and engineers working on the topic.
The hope is based around alignment, a term used broadly to mean the degree to which an AGI can be made to conform to human goals and objectives. Poorly aligned AGI would probably be diasastrous and unsafe, well aligned AGI might be beneficial and safe. And let’s be clear at the outset: there are concerns about how AGI (and even current technology like ChatGPT) will cause disruption and harm by affecting social behaviour or employment in the way new technologies have done in the past. But that’s not what we are addressing here. The more significant thinking is about the existential threat to humanity. Will an AGI spell the end, will it render humans extinct?
Nor are we discussing here whether or not it will be possible to develop an AGI, some people argue not, that there’s some kind of unbridgeable leap between current AI systems and an intelligent system with thoughts, ideas of its own, and self-awareness. But we know this is incorrect; such a system has already been created by evolution – humans! This level of intelligence is achievable and it’s already been done. It may take time, it might not be easy, but developments in neural networks and related systems will lead to AGI sooner or later. ChatGPT can already write workable computer code – just think about that for a moment.
The hopeful news comes from OpenAI, the company that created ChatGPT and the GPT software behind it. They have found that process supervision produces better results than outcome supervision. And this gives us a much better chance of understanding how the AI makes its choices. Process supervision feeds back on the quality of the interim stages of an AI’s processing; but outcome supervision feeds back on only the quality of the final result.
Understanding how an AGI works might help developers build in robust alignment features; and if process supervision is more effective than alternatives, it stands a good chance of being employed by developers.
The unhopeful news comes from discussions about the nature of risk and human thinking about risk, expressed in discussions on LESSWRONG and elsewhere.
Currently, only a very small number of people are concerned about a possible existential threat from AGI. But most of the people with that concern are some of the same people that have knowledge and experience of AGI, what AI can currently do, and how quickly the systems might advance and escape our control.
Personally, I think the threat is real and potentially impossible to stop with very little time for effective countermeasures. But I also think it might just be possible to avoid the danger, but only if we have binding international agreements and strong oversight. However, I further suspect that it might be impossible to agree those necessary measures completely enough, rigorously enough, or fast enough to be fully effective.
In my judgement we are on rather shaky ground, and the more people who give this serious thought as soon as possible, the better. When the stakes are so very high there can be no such thing as too careful or too thorough.
For such an alarming topic it’s a remarkably calm discussion, but also a very informative and thought-provoking conversation.
The rise of AI and the possibility (some would say certainty) that this means the end of human civilisation and the extinction of our species, are topics being seriously warned against by a number of thinkers, scientists, and AI experts.
The Guardian discusses these issues with ‘the father of AI’, Geoffrey Hinton.
Listen to the podcast and see what you think. The potential end of humanity is not something to be swept under the carpet. We need to think about it right now. It would be extraordinarily foolish to wait, it might already be too late. For such an alarming topic it’s a remarkably calm discussion, but also a very informative and thought-provoking conversation.
I believe everyone should have the chance to listen to this.
Prepare for much better and more natural interaction with devices like Alexa, your satnav, book readers, aids for the blind and much more. No more need to remember the right words and phrases to use.
What does a conversation with ChatGPT look like? I test it out below and add some short notes and thoughts on what it might mean for the future.
I ran a simple chat session with ChatGPT, just to get an idea of how good it is at understanding questions and returning understandable, useful answers in ordinary language. Everything in the conversation below with a yellow background was typed by me, the rest is from ChatGPT. Where my typing is an aside for you, my readers, and not sent to ChatGPT, I’ve put it in braces {}.
The conversation is quite long, but I hope it will interest you and perhaps cause you to raise questions in your own mind. If you want to discuss any thoughts on AI in general (and ChatGPT in particular) just leave them in the comments section below. Thanks.
Chris – Hi, my name is Chris and I’d like you to introduce yourself; please don’t be nervous, just give me an informal introduction.
ChatGPT – Hi Chris! I’m ChatGPT, a language model designed to communicate with humans and assist with various tasks. Think of me as a robot that can talk with you through text. I’m always eager to learn new things and help you in any way I can, so feel free to ask me anything!
Chris – Thanks! Good to meet you. OK, I’ve got the idea that you are robot-like. There are a couple of things I’d like to know. Why are you called ‘ChatGPT’? And what is a language model?
ChatGPT – Good to meet you too, Chris!
ChatGPT is named after the GPT (Generative Pre-trained Transformer) language model architecture, which is used to build me. GPT is a type of neural network that is trained on massive amounts of text data, allowing it to generate coherent and natural-sounding text. The “Chat” part of my name refers to my ability to chat with humans like you, using text as our mode of communication.
Speaking of language models, a language model is a type of artificial intelligence that can analyze and generate human language. Essentially, a language model is a computer program that is trained on large amounts of text data, allowing it to learn the rules and patterns of language. Once trained, a language model can perform a wide variety of tasks, such as generating text, translating between languages, or answering questions. I’m a particularly advanced type of language model, based on the GPT-3.5 architecture, which allows me to handle a wide range of tasks and communicate with people like you in a natural and engaging way.
Chris – I must congratulate you, your replies are certainly engaging and natural. Do you mind if I ask some more questions? This is an interesting conversation.
ChatGPT – Thank you, Chris! I’m always happy to answer questions and have interesting conversations, so please feel free to ask me anything you’d like.
{I thought I’d introduce some typos next, to see if ChatGPT would struggle to parse the text. But it didn’t bat an electronic eyelid, answering quickly and correctly.}
Chris – What sort of toxt deta were you trainers with? Do you know where it came from? A few examples of sources would be helpful.
ChatGPT – I was trained on a diverse set of text data, ranging from news articles and academic papers to social media posts and online discussions. The training data comes from various sources and domains, covering a wide range of topics and writing styles to help me understand and generate text in a natural and diverse manner.
Some examples of sources that were used to train me include:
The training data was collected from various sources and pre-processed to remove irrelevant or duplicated content, ensuring that the data was of high quality and diverse enough to train me to generate human-like text.
Chris – In that case let’s try to draw on some of the material you’ve seen from those sources. I live in an English town called Cirencester. It’s in a region called The Cotswolds. Please write a short essay about Cirencester for me.
ChatGPT – Sure, I’d be happy to write a short essay about Cirencester for you!
Cirencester is a charming town located in the heart of the Cotswolds, a picturesque region known for its rolling hills, quaint villages, and traditional architecture. With a history dating back to Roman times, Cirencester is home to a wealth of historical and cultural attractions that draw visitors from around the world.
One of the town’s most notable landmarks is the Cirencester Parish Church of St. John Baptist, a stunning example of medieval architecture that has been standing for over 900 years. The church is home to a variety of beautiful stained-glass windows, intricate stone carvings, and historical artifacts that provide a glimpse into the town’s rich history.
Another must-visit attraction in Cirencester is the Corinium Museum, which houses an impressive collection of Roman artifacts and exhibits that showcase the town’s Roman heritage. Visitors can explore the museum’s extensive collection of mosaics, coins, and pottery, as well as learn about the daily lives of the Roman citizens who once called Cirencester home.
In addition to its historical attractions, Cirencester is also known for its natural beauty and outdoor activities. The town is surrounded by idyllic countryside, with plenty of opportunities for hiking, cycling, and exploring the Cotswold Way, a 100-mile walking trail that winds through the region’s stunning landscapes.
Whether you’re interested in history, culture, or outdoor activities, Cirencester has something to offer. With its unique blend of ancient and modern, this charming town is a must-visit destination for anyone looking to experience the best of the Cotswolds.
Chris – That was really good! Do you write poetry as well? How about a short poem (three or four verses) about Cirencester?
ChatGPT – Sure, I can certainly give it a try! Here’s a short poem about Cirencester:
In the heart of the Cotswolds lies a town,
With Roman roots and ancient renown.
Cirencester, where history meets charm,
And natural beauty keeps us calm.
Streets lined with stone and buildings grand,
A parish church that still does stand.
Corinium’s past in the museum’s keep,
A town that makes our hearts leap.
Cirencester, oh, how you inspire,
In every season, by every fire.
A gem in the Cotswolds, a treasure to see,
A place where our hearts are free.
Chris – Great! Here’s something a bit harder, can you do a haiku on Cirencester?
ChatGPT – Sure, here’s a haiku about Cirencester:
Roman walls stand tall,
Cirencester’s ancient embrace,
Cotswold’s heart of stone.
Chris – Great again, and hard to do. I have one last question for you; how should I solve a quadratic?
ChatGPT – To solve a quadratic equation, there are several methods, but one of the most commonly used is the quadratic formula. The quadratic formula states that for an equation of the form ax^2 + bx + c = 0, the solution for x is given by:
x = (-b ± sqrt(b^2 – 4ac)) / 2a
where “sqrt” denotes the square root function.
To use this formula, you simply plug in the values of a, b, and c from your quadratic equation into the formula, and then simplify the expression to find the solutions for x.
It’s worth noting that there are other methods for solving quadratic equations, such as factoring and completing the square, but the quadratic formula is generally considered the most reliable and efficient method, especially for more complex equations.
Chris – Thanks! Enjoy your time chatting to some other people. By the way, how many people can you converse with at the same time? I can only manage four or five.
ChatGPT – As an AI language model, I can theoretically converse with an unlimited number of people simultaneously, as long as the incoming messages are not too frequent and I have enough computing resources to process them all in a timely manner. However, I can only provide individualized responses to each person based on the messages they send me, so the more people involved in the conversation, the more difficult it may be to keep track of multiple threads of conversation at once.
I think you’ll agree that ChatGPT is pretty good at giving understandable and useful responses in English to a range of questions. It’s pretty much the sort of conversation you’d expect from a very well-read human. It seems most unlikely that today’s AI systems, including ChatGPT, actually ‘think’ in anything like a human way. It manipulates language very cleverly in ways that are useful and seem human-like. But these AI systems don’t understand meaning, they don’t have feelings or emotions, they don’t have thoughts or opinions.
Might they have the earliest glimmerings of these things? There’s no evidence that they do. But the teams developing such systems have sometimes expressed caution. The danger here is that any much more advanced future system that did have a spark of self-awareness might arise without us even knowing about it, and if it was capable of modifying itself it could become exponentially more clever. Even a few days of exponential improvement could result in a more powerful mind than any human. After that, the AI could outwit us with ease and make decisions that we might not like. The difficulty is that the time available for us to intervene would be a much smaller window than we might suppose or plan for.
In the near future (likely starting in months, not years) a welcome development will improve those systems that already use speech recognition and/or speech generation. Prepare for much better and more natural interaction with devices like Alexa, your satnav, book readers, aids for the blind and much more. No more need to remember the right words and phrases to use, fewer misunderstood commands, the ability to ask for more or different information.
Expect help with programming, planning astronomical observing sessions, writing up research, having your articles proof read and copy edited. Perhaps soon you’ll be able to throw away your keyboard! Laptops might start to look like tablets. Bedtime? Ask for a story.
ChatGPT can do that already…
Chris – Write me a children’s bedtime story about a dog, a mouse, an oak tree and an ant hill.
ChatGPT – Once upon a time, there was a friendly dog named Max. Max loved exploring the forest near his home and making new friends. One day, while he was wandering through the woods, he met a tiny mouse named Mimi. Max and Mimi quickly became best friends and spent hours playing together.
As they were playing, Max noticed a beautiful oak tree nearby. The tree was enormous, with branches stretching high into the sky. Max and Mimi decided to climb up the tree and see what they could find. When they reached the top, they discovered a bustling ant hill. Thousands of tiny ants were working together to gather food and build their home.
Max and Mimi watched in awe as the ants worked tirelessly. They were amazed at how much the little ants could accomplish when they worked together. Max and Mimi decided to help the ants by gathering food for them. Max used his strong nose to sniff out berries, while Mimi used her quick reflexes to catch insects.
The ants were overjoyed to receive Max and Mimi’s help. They thanked the two friends by giving them a special gift – a magical acorn that would grant them one wish. Max and Mimi thought long and hard about what they should wish for.
Finally, Max said, “I wish for all the animals in the forest to be friends, just like us.” Mimi nodded in agreement. She added, “And I wish for the oak tree to grow even taller and stronger, so it can provide shelter for even more creatures.”
Their wish was granted, and from that day forward, all the animals in the forest were friends. The oak tree grew taller and stronger, and even more animals found shelter in its branches. Max and Mimi continued to play together and explore the forest, knowing that their wish had made a difference in the lives of so many creatures.
As the sun began to set, Max and Mimi said goodnight to their new friends and curled up together under the oak tree. The stars twinkled above them, and the gentle breeze rustled the leaves. Max and Mimi felt grateful for the adventure they had shared and fell asleep with happy hearts, dreaming of all the fun they would have tomorrow.
It can write essays, explain more or less anything you might ask, it can even pass many written exams on all kinds of topics
If you’ve been paying attention to the technology scene recently, you cannot fail to have heard of ChatGPT. What is it? Quite simply it’s a chatbot, a type and read interface for version 3.5 of the GPT engine developed by Open AI.
If all that sounds confusing, a chatbot is artificial intelligence (AI) software, in this case presented as a webpage, where you can have a conversation with a computer program. You type in a question and the chatbot sends a reply. Repeated questions and replies form a conversation, in ChatGPT’s case a remarkably smooth, almost ‘human’ experience.
If you haven’t yet tried it, then you really should. It’s free and very easy to use. I encourage you to visit chat.openai.com, sign up for a free account, and type a question. There are no rules about what you can and cannot type; ChatGPT is accomplished at understanding ordinary language and gives good, conversational replies.
Assuming you tried it out and have found your way back to my blog post, I’d like to explain a bit more about it. ChatGPT was released as a website at the end of November in 2022 and has grown in popularity very, very fast. If you tried it out for yourself, I’m sure you will understand why. It’s compelling, it can write essays, explain more or less anything you might ask, it can even pass many written exams on all kinds of topics. And version 4.0 of the engine is already available as a paid option and is far more capable. It can take images as input as well as text and has, for example, built a working website based on a sketch and a description of what the website should do. That is little short of astounding!
The company, Open AI, was created to work on artificial intelligence with safety very much in mind. Sometimes, ChatGPT generates false answers; that can be an issue but it is not deliberate. What if more advanced AI became able to reason as people do, what if it started to think and develop it’s own goals, and what if its intellect became faster and more nimble than our own? Could we prevent it from taking over? Would it be benign, or might it become hostile? Would we be able to control it? These are serious issues. We need to think these things through now, before it becomes too late.
I don’t want to be alarmist, and AI as we currently experience it is far from becoming a threat. It may prove useful in many ways and we’ll see that begin to happen very soon. But we’ll need to manage it in ways that prevent it helping people do bad things. We don’t want such technology to enhance criminal activity, for example. So there’s a great deal to consider right now, and the need for that will only increase as AI systems become more and more capable. For more on this I recommend Sean Carroll’s podcast episode 230, linked below.
I’m going to close this blog post at this point, but I’ll be back soon with a sort of interview with ChatGPT. I’ll ask some questions, let the software answer, and publish the conversation.
Meanwhile, have some ChatGPT conversations for yourself and see what you think.
Journeys of Heart and Mind 