The Limits of AI Training

The Limits of AI Training — and the Race to Build “AI That Grows Up”

There are two major debates in AI training today, and they point to two very different futures:

The Current Approach

Once an LLM is trained, its “brain” is more or less fixed. Adding new knowledge is extremely limited; techniques like RL only patch tiny bits. If you want the model to be smarter, you have to retrain it from scratch.

Imagine “creating” a 5-year-old child. After a few years you want a 10-year-old? You need to create a new child — or borrow one. Want a 15-year-old? Repeat.

Obviously, that’s inefficient. What we really want is to create a 5-year-old and let them learn, grow, and naturally become 10, then 15. Nobody wants to manufacture a new child every five years just because the old one is outdated.

The Desired Approach Many top-tier labs are exploring this direction.

The goal is to train a powerful base model once, then release it to the world. As it operates, the model continuously learns, improves, and upgrades its own cognitive abilities.

Occasionally, you apply incremental training directly to the running model, instead of replacing it with a completely new one. In other words: create an “AI baby” and let it grow — rather than producing a new one every year and dropping the old one.

Where We Are Today

All commercial AI companies are still using approach (1): produce a new model every year, retire the previous one. There’s still room to optimise, but growth potential will hit a ceiling within a few years.

A few reputable labs are pursuing approach (2), but no breakthrough has emerged yet — though enormous amounts of funding are flowing into this search.

If this new approach succeeds, training costs could drop dramatically. Incremental training is much cheaper than full retraining, and it would finally allow models to accumulate intelligence continuously instead of being rebuilt from scratch every cycle.

The next frontier of AI isn’t just “bigger models.” It’s models that can learn like humans — continuously, without resets, without starting over every year.

Written by Steven Nguyen

Author

Ai Base Network (ABN), ABN ASIA was founded by people with deep roots in academia, with work experience in the US, Holland, Hungary, Japan, South Korea, Singapore, and Vietnam. ABN Asia is where academia and technology meet opportunity. With our cutting-edge solutions and competent software development services, we're helping businesses level up and take on the global scene. Our commitment: Faster. Better. More reliable. In most cases: Cheaper as well.

Feel free to reach out to us whenever you require IT services, digital consulting, off-the-shelf software solutions, or if you'd like to send us requests for proposals (RFPs). You can contact us at [email protected]. We're ready to assist you with all your technology needs.