Category: Technology

AGI – Hopeful and less hopeful news

Personally, I think the threat is real and potentially impossible to stop with very little time for effective countermeasures.

It’s fair to say that few people are currently thinking about the future of Artificial General Intelligence (AGI). Those who do are mostly researchers and engineers working on the topic.


Wikipedia article – Existential risk from artificial general intelligence

The hope is based around alignment, a term used broadly to mean the degree to which an AGI can be made to conform to human goals and objectives. Poorly aligned AGI would probably be diasastrous and unsafe, well aligned AGI might be beneficial and safe. And let’s be clear at the outset: there are concerns about how AGI (and even current technology like ChatGPT) will cause disruption and harm by affecting social behaviour or employment in the way new technologies have done in the past. But that’s not what we are addressing here. The more significant thinking is about the existential threat to humanity. Will an AGI spell the end, will it render humans extinct?

Nor are we discussing here whether or not it will be possible to develop an AGI, some people argue not, that there’s some kind of unbridgeable leap between current AI systems and an intelligent system with thoughts, ideas of its own, and self-awareness. But we know this is incorrect; such a system has already been created by evolution – humans! This level of intelligence is achievable and it’s already been done. It may take time, it might not be easy, but developments in neural networks and related systems will lead to AGI sooner or later. ChatGPT can already write workable computer code – just think about that for a moment.

The hopeful news

The hopeful news comes from OpenAI, the company that created ChatGPT and the GPT software behind it. They have found that process supervision produces better results than outcome supervision. And this gives us a much better chance of understanding how the AI makes its choices. Process supervision feeds back on the quality of the interim stages of an AI’s processing; but outcome supervision feeds back on only the quality of the final result.

Understanding how an AGI works might help developers build in robust alignment features; and if process supervision is more effective than alternatives, it stands a good chance of being employed by developers.

The less hopeful news

The unhopeful news comes from discussions about the nature of risk and human thinking about risk, expressed in discussions on LESSWRONG and elsewhere.

Closing thoughts

Currently, only a very small number of people are concerned about a possible existential threat from AGI. But most of the people with that concern are some of the same people that have knowledge and experience of AGI, what AI can currently do, and how quickly the systems might advance and escape our control.

Personally, I think the threat is real and potentially impossible to stop with very little time for effective countermeasures. But I also think it might just be possible to avoid the danger, but only if we have binding international agreements and strong oversight. However, I further suspect that it might be impossible to agree those necessary measures completely enough, rigorously enough, or fast enough to be fully effective.

In my judgement we are on rather shaky ground, and the more people who give this serious thought as soon as possible, the better. When the stakes are so very high there can be no such thing as too careful or too thorough.

See also:

The end of humanity?

For such an alarming topic it’s a remarkably calm discussion, but also a very informative and thought-provoking conversation.

The rise of AI and the possibility (some would say certainty) that this means the end of human civilisation and the extinction of our species, are topics being seriously warned against by a number of thinkers, scientists, and AI experts.

Artificial General Intelligence – Image from Wikimedia

The Guardian discusses these issues with ‘the father of AI’, Geoffrey Hinton.

Listen to the podcast and see what you think. The potential end of humanity is not something to be swept under the carpet. We need to think about it right now. It would be extraordinarily foolish to wait, it might already be too late. For such an alarming topic it’s a remarkably calm discussion, but also a very informative and thought-provoking conversation.

I believe everyone should have the chance to listen to this.

See also:

Blast from the past… 6

In May 1993 we made some very large bubbles in the back garden.

< Previous | Index | Next >

Notes from bygone years – May without limits

A year ago

In May 2022 we visited St Neots in Cambridgeshire, our home town before moving to Cirencester where we live today. We were visiting friends for a big party in their local village hall.

Pretending to be on a tropical beach

The whole thing was themed for a Caribbean feel, and we all had our photos taken against a backdrop of palm fronds, a golden beach, and sparkling sea dotted with distant islands.

<Apr 2022 – Jun 2022>

Two years ago

In May 2021 I took a photo of this plaque in Black Jack Street, Cirencester. It commemorates 1900 years of the town’s existence, founded by the Roman invaders in 75 AD. I wonder what they’d make of it now?

The phoenix rising from the ashes has long been Cirencester’s emblem. Corinium, as the Romans called it, was a frontier settlement in 75 AD and the years immediately following, but it grew to become the regional capital of south western Britain – the province of Britannia Prima.

Wikipedia has a good article about Corinium.

<Apr 2021 – Jun 2021>

Five years ago

In May 2018 I wrote about the work going on in our new house. The builders had done much of the work on the new extensions, but nothing was finished yet and a lot of our possessions were inaccessible – including our summer clothes. And the weather was getting warmer and warmer!

Our partly liveable house

If you’d like to know more, read the original article.

<Apr 2018 – Jun 2021>

Ten years ago

In May 2013 I was reading ‘The Shaping of Things to Come’, an excellent book by Michael Frost and Alan Hirsch. Amongst other things, they remark on the fact that Jesus was a Jew, the New Testament was written almost entirely by Jewish authors (the exception being Luke), and the earliest church (in Jerusalem) was Jewish in nature too. If we are to understand Jesus, the New Testament, and the church, we have to pay attention to their Jewishness. Read the original post.

<Apr 2013 – Jun 2013>

Fifteen years ago
The ‘Sagrada Familia’

May 2008 – We had a holiday in Catalonia and visited Barcelona where we took a look at the famous Sagrada Familia with its amazing ‘biological’ shapes. What an astonishing place it is!

To learn more about this wonderful feat of design and engineering by the architect Antoni Gaudi, read the Wikipedia article.


<Apr 2008 – Jun 2008>

Twenty years ago

In May 2003 I was reminded about the significance of rivers and their behaviour and about the river in the books of Ezekiel and Revelation. Life is not about the mundane things we do from day to day, and we don’t get far by our own abilities. Real life has an essentially spiritual focus. I wrote a blog post about a meeting that helped me see these things more clearly.

<Apr 2003 – Jun 2003>

Twenty-five years ago

May 1998 saw me designing an automated news website for internal use by research teams at Unilever. I’d only been in the job for a month, and I already had an overall mechanism in mind and was working on the programming required.

<Apr 1998 – Jun 1998>

Thirty years ago

In May 1993 we made some very large bubbles in the back garden. We had a special bubble-making kit with a large fabric loop on the end of a wand. It was an amazing process! The picture shows my daughter Beth and nephew Tom experimenting with a bubble.

<Apr 1993 – Jun 1993>

Thirty-five years ago

In May 1988 I was working at Long Ashton Research Station near Bristol, developing techniques for imaging water droplets on waxy leaves by rapid freezing and scanning electron microscopy. This led to a paper on the topic.

<No earlier data – Jun 1988>

Forty years ago

It’s May 1983 and time for the annual school fete; my daughter Beth and her friend Vicky were in fancy dress, walking with their teacher.

How time flies – my grandchildren are older now than my children were then.

<No earlier data – Jun 1983>

Forty-five years ago

In May 1978 I borrowed a friend’s car (we didn’t have a car at the time) and drove with a very excited daughter to visit her baby sister at Bristol Maternity Hospital.

<No earlier data – Jun 1978>

Fifty years ago

In May 1973 Mum and Dad visited us for the day and we spent some time in Clevedon. Here’s Dad taking photos on the rocky shoreline.

<No earlier data – Jun 1973>

Fifty-five years ago

In May 1968 I was in lodgings in Pershore, working on a fruit and vegetable farm nearby as part of my undergraduate degree at Bath University. It was a sandwich course in horticulture, and the summers were the industrial experience part of the course.

<No earlier data – Jun 1968>

Sixty years ago

May 1963 and I was in my fifth year at Cirencester Grammar School.

<No earlier data – Jun 1963>

< Previous | Index | Next >

Starship launch attempt postponed

** LATEST ** – The next launch attempt is now scheduled for Thursday 20th April. Check the latest news at Spaceflight Now.

Monday’s planned test flight of the Super Heavy booster carrying Starship was called off due to a pressurisation issue caused by a failed valve. The booster’s ability to carry a significant load (like a fully fuelled Starship) depends on the fuel tanks being pressurised at all times. It’s the same process that makes drinks cans so strong despite being very thin walls. An empty can crumples whereas an unopened can is pressurised and is significantly stronger.

If the valve issue can be fixed on Tuesday, there might be a new launch attempt on Wednesday. Watch this space (no pun intended).

See also:

Starship’s first launch

Just a short post this time, to alert anyone who may not know… SpaceX is planning the first ever launch of Starship on an orbital flight tomorrow, Monday 17th April.

There are several places you can watch this attempt live on YouTube. I recommend three of them in particular. If you’re new to all this, try the SpaceX stream. If you’re more knowledgeable you might prefer one of the others.

  • Felix Schlang, a German who moved to Florida to be closer to the action
  • Marcus House, an Australian spaceflight enthusiast in Tasmania
  • SpaceX, the company’s official stream

I just wanted to get this alert out today. I’ll be back after the launch attempt with more thoughts. (More details about Super Heavy and Starship).