Well... to give people some context:

Hi, I’m Leopold Aschenbrenner.

I recently founded an investment firm focused on AGI...

Note: AGI = Artificial General Intelligence = a full artificial consciousness, the same as a person.

1. The guy writing the paper everyone is quoting is just a finance guy, pumping up his own investment. He is by no means an expert in the field of machine learning.

2. He is simply making linear extrapolations of the progress in machine learning LLMs. Nothing in nature is truly linear (although it can look that way sometimes).

3. He glosses over several major problems, that are effectively show-stoppers, with some "hand waving" explanations.

It is important to remember that machine learning (referred to as Artificial Intelligence for hype) is only able to recognize patterns in data. Really. That's it. Machine learning has literally zero comprehension, and does not understand the meaning of a single word used for inputs or outputs. It is only trained to recognize patterns in content created by people.

I am not saying AGI is impossible, but ChatGPT4 is nowhere near true AGI.

As an example, he makes a big deal about test scores in math tests. The way an LLM is trained, it is given all of the questions, along with all of the correct answers to the questions as part of its training. The inputs must be categorized and tagged by people, and the results produced by the LLM must then be scored by people (which is how the system "learns" the difference between good answers and bad answers). People must do all of these tagging and scoring tasks manually, because the LLM has no intelligence or comprehension regarding any input or output.

So, for the math test example, the LLM would find questions matching the test question (pattern recognition), and supply the associated correct answer that was provided as part of its training. The LLM did not understand one bit of either the test question, or the answer. It is all based solely on pattern recognition.

So, this paper being quoted is from an investment guy pumping his own investment... which he does not really understand very well.

What are the show-stopper problems? First, the companies pushing machine learning are running out of human created content they can use to train their LLMs. They have resorted to stealing a lot of data, and the lawsuits against those thefts of intellectual property have only just begun recently.

Some are instead trying to have one LLM generate content that can then used to train another LLM. This has proven not to work in practice (and worst case can destroy the whole model). So, they are already running out of training data after scanning the entire Internet, and cannot generate their own training data. What will they do? Nobody knows. They have no answer to this problem.

Second, the expense of running and training LLMs vs. the revenue they generate. Using ChatGPT4 as an example, the cloud infrastructure to run it and store all the data costs roughly $250 Million per year. The cost of the army of people to categorize and tag all inputs, and score all outputs is also enormous... and neither of those expenses ever ends. Training is never finished. It is infinite. As for revenue to pay for all of those massive expenses, realistically there is none... or very little.

Remember the DotCom bubble in the 90's? Remember all the companies that burned through so much money and produced no profits, ever? Yeah, A.I. is kind of like that, all over again.

Beware the A.I. hype. Don't totally ignore A.I., but realize that a lot of the hype comes from people that are out to make as much money from gullible people as they can, before the bubble bursts. Sadly, you see a lot of the same sort of hype from the Bitcoin Bros... just as a heads up.