From Reuters:

Exclusive: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster – sources

By Anna Tong, Jeffrey Dastin and Krystal Hu
November 22, 2023 4:56 PM PSTUpdated an hour ago

Nov 22 (Reuters) – Ahead of OpenAI CEO Sam Altman’s four days in exile, several staff researchers sent the board of directors a letter warning of a powerful artificial intelligence discovery that they said could threaten humanity, two people familiar with the matter told Reuters.

OpenAI started out as not for profit that then got transformed, hesto presto, into a hot property. But the board remained dominated by not-for-profit people worried about Eliezer Yudkowsky’s terrors of AI become super-intelligent and killing us all like in Terminator.

Personally, I don’t have much of an opinion on this subject, other than it’s not surprising that Microsoft wound up getting Sam Altman reinstated as CEO of OpenAI. Money talks.

The previously unreported letter and AI algorithm was a key development ahead of the board’s ouster of Altman, the poster child of generative AI, the two sources said. Before his triumphant return late Tuesday, more than 700 employees had threatened to quit and join backer Microsoft (MSFT.O) in solidarity with their fired leader. …

According to one of the sources, long-time executive Mira Murati mentioned the project, called Q*, to employees on Wednesday and said that a letter was sent to the board prior to this weekend’s events….

The maker of ChatGPT had made progress on Q* (pronounced Q-Star), which some internally believe could be a breakthrough in the startup’s search for superintelligence, also known as artificial general intelligence (AGI), one of the people told Reuters. OpenAI defines AGI as AI systems that are smarter than humans.

Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because they were not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said.

Reuters could not independently verify the capabilities of Q* claimed by the researchers.

SUPERINTELLIGENCE
Researchers consider math to be a frontier of generative AI development. Currently, generative AI is good at writing and language translation by statistically predicting the next word, and answers to the same question can vary widely.

In other words, ChatGPT is a good BS artist.

But conquering the ability to do math — where there is only one right answer — implies AI would have greater reasoning capabilities resembling human intelligence. This could be applied to novel scientific research, for instance, AI researchers believe.

… There has long been discussion among computer scientists about the danger posed by superintelligent machines, for instance if they might decide that the destruction of humanity was in their interest.

Beats me …