“

The AI field is flooded with capital, computing power, and data, but mainstream research methods may hinder genuine breakthroughs.

”

Author | Zheng Huimin Editor | Zuri At just 20 years old, he achieved financial freedom through the success of several groundbreaking 3D games.

With ample funds, he ventured into rocketry, unfortunately ending in failure.

Fascinated by VR, he joined Oculus, a one-year-old company, as CTO, and within a year, Oculus soared, eventually being acquired by Meta for $2 billion.

As a man who has rejected offers from Musk, Gates, Jobs, and Sam Altman,

In the era of AGI, he has decided to re-emerge and make a big splash.

In 2022, he founded Keen Technologies, taking an unconventional path to achieve AGI, securing $20 million in investment from Sequoia Capital, the former CEO of GitHub, the co-founder of Stripe, the co-founder and CEO of Shopify, and others.

He is the programming genius born in the 1970s, known as "God of Code," John Carmack.

01

The Path to Godhood

As a child, Carmack was no angel.

Addicted to games and computers, he even resorted to theft to satisfy his craving for computers, leading to a stint in a juvenile detention center. In high school, his unruly nature earned him a year-long suspension.

Naturally intelligent, he graduated with a 4.0 GPA upon returning to school and entered the prestigious public research university of the University of Kansas.

However, after just two semesters, Carmack grew weary of the regimented college life and dropped out to enter the workforce.

From then on, Carmack lived his dream life, meeting like-minded partners—programmer John Romero, game designer Tom Hall, and artist Ardian Carmack. The four worked day and night, coding and developing games, releasing the first installment of the "Commander Keen" series, which became a hit. The name of Carmack's AGI company, Keen Technologies, is derived from this series.

The success of their works sparked the idea of entrepreneurship, and in 1991, the four officially founded the renowned game company id Software.

In 1992, id released "Wolfenstein 3-D," the world's first 3D first-person shooter (FPS), pioneering this unprecedented game genre. With its 3D and shooting concepts, it revitalized a work from the 1980s.

The following year's "Doom" achieved even greater success, becoming the king of FPS games with its fast-paced combat, violent visuals, and exceptional compatibility. It solidified id's position at the pinnacle of the industry, generating over a billion dollars in profits within a year and boosting sales of then-expensive PCs.

Image source: PCGamesN

Faced with immense profits, Carmack made a surprising move by releasing part of "Doom's" source code to the public, allowing loyal fans to modify levels and graphics, essentially granting them the opportunity for secondary creation, which further ignited players' enthusiasm.

In 1995, "Doom" even outpaced Windows 95 in installations, prompting Bill Gates to consider acquiring id.

After being rejected by Carmack, Microsoft had to port "Doom" to its system on its own. Gates even appeared in TV ads, using the game to promote Windows.

These popular masterpieces elevated Carmack to godhood in the hearts of gamers.

The 3D graphics display technology and game engine used by Carmack in his games later inspired the birth of Epic Games' Unreal Engine and Unity 3D development tools, profoundly impacting the game industry.

There's a side note here. Carmack and Jobs once had a sweet relationship, becoming acquainted early in id's founding. Carmack even convinced Jobs to use open-source OpenGL as the 2D and 3D graphics rendering engine for Apple computers, which he called "one of the biggest indirect impacts on the PC industry."

Jobs and Carmack | Image source: Variety

But even with Jobs, Carmack's independent streak wouldn't budge. It's said that Jobs once wanted Carmack to postpone his wedding to give a keynote speech, but Carmack refused.

Getting back to the main story, the best-selling games also earned Carmack a fortune at a young age. With money no object, he turned his attention to the highly expensive civilian aerospace field.

In 2000, he founded Armadillo Aerospace to develop cutting-edge space exploration projects, where he met Musk, who shared a passion for outer space.

Image source: IBTimes UK

The two admired each other, and after Carmack's space venture hit a dead end, Musk tweeted an invitation for Carmack to join SpaceX in 2015. However, like with Jobs, Carmack declined, as he was fully immersed in VR at the time.

Later, reflecting on his unfinished space venture, Carmack attributed part of the failure to not giving it his all, noting that Musk's all-in approach to his endeavors was worth learning from. This made him decide to go all-in on whatever he pursues in the future.

Image source: Twitter, International Business Times

In 2013, Carmack left id Software, which he founded, to join Oculus as CTO, which was still in its infancy. On his first day, the influx of fans caused Oculus's blog system to crash. In 2014, Oculus was acquired by Meta for $2 billion, and Carmack joined Meta to continue his VR research.

Carmack was dedicated to his work: he personally oversaw company projects, collaborating with Samsung to develop Gear VR, shouldering a workload that would typically require an entire team. From the PC-tethered Oculus Rift to the early VR all-in-one Oculus Go, to the cost-effective Oculus Quest, and Oculus's app store ecosystem, Carmack's presence was felt everywhere. He was also warm towards players: to enable them to play "Minecraft" in VR, he spent over a year porting the game to the Oculus store on his own.

However, Carmack's focus on technology and down-to-earth approach clashed with Meta's aggressive metaverse strategy, leading to his gradual loss of influence in Oculus's major decisions. Carmack's passion for VR was also eroded by Meta's slow operational efficiency. Eventually, he chose to leave upon contract expiration.

02

The Call of AGI: Carmack Enters Artificial Intelligence

This time, it's AGI that has turned the god's head.

While serving as Oculus CTO, the three co-founders of OpenAI—Sam Altman, Greg Brockman, and Ilya Sutskever—persuaded Carmack to join OpenAI, but the independent-minded Carmack unsurprisingly declined.

However, Carmack did read over 40 related research papers listed by Ilya, catching up on AI algorithms from scratch. Despite Python being the mainstream programming language, he wrote a convolutional neural network in C++ in just a week of seclusion.

In 2022, Carmack officially entered the fray, founding the AGI company Keen Technologies. Unlike large companies that assess commercial prospects before investing in a new technology field, Carmack bluntly stated that his current work had zero short-term commercial opportunities.

Image source: John Carmack X Account Keen's initial investors are formidable: including GitHub's former CEO Nat Friedman, former Apple AI executive Daniel Gross, Stripe co-founder Patrick Collison, Shopify co-founder Tobi Lutke, Sequoia Capital, and legendary Silicon Valley chip architect Jim Keller.

Among them, Nat Friedman and Daniel Gross were early investors in AI unicorns like Cohere and Character.ai, backing half of America's AI landscape. Daniel Gross recently co-founded an AI safety company founded by former OpenAI chief scientist Ilya. Jim Keller has over 20 years of chip design experience, hailed as the father of the Zen architecture, and has led chip design and development at companies like Apple, AMD, and Tesla.

Carmack, who had already achieved financial freedom through game sales, didn't need the money. Accepting external funding was merely to give himself a sense of urgency in entrepreneurship.

For this venture, Carmack brought on board Richard Sutton, the father of reinforcement learning.

Sutton is the co-author of the classic textbook "Reinforcement Learning: An Introduction," known as the godfather of modern reinforcement learning. He participated in AlphaGo research at DeepMind and currently teaches at the Alberta Machine Intelligence Institute in Canada.

Carmack and Sutton | Image source: Taproot Edmonton

Sutton wrote a famous essay, "The Bitter Lesson," which has become a must-read for OpenAI engineers. The essay emphasizes the importance of increased computing power for AI development, aligning with OpenAI's approach at the time and validated in the realization of large language models.

Through email exchanges, Carmack and Sutton discovered that both wanted to take an "unconventional" path.

In Carmack's words, "The AI field is flooded with capital, computing power, and data, but mainstream research methods may hinder genuine breakthroughs. Sutton and I just happen to stand outside the mainstream."

They both believe that the ultimate AGI won't be overly complex. Compared to current large-scale applications with millions of lines of code, which one person could never finish, AGI code could be written by a single person.

"If you take the entire human DNA, its information content is less than 1GB, and the brain is just a small part of it—about 40MB, and it's not strictly coded. This suggests that the coding that constitutes our brain's intelligence isn't that much," said Carmack.

In Carmack's vision, AGI should maintain continuous online learning, rather than having most parameters remain unchanged after pre-training, as is currently the case.

However, the specific path Keen Technologies will take remains a mystery. Visiting the company's website reveals only a simple statement: "John Carmack's AGI attempt, with undisclosed information."

03

The 2030 Appointment: How Can Small AI Companies Score Big?

Although everything is still shrouded in mystery, at least one thing is clear: Carmack believes we have a chance to glimpse AGI by 2030.

"I can see the destination, but the path between the current situation and the destination is full of uncertainties. No one knows how to achieve it, but I have confidence that we will arrive, or at least someone will, and the time point is likely 2030."

According to reports, Keen's founding technical team comprises only four people, and its funding remains at the initial $20 million. Neither its talent nor capital reserves match those of leading AI unicorns like OpenAI or giants like Google and Meta.

Image source: TechCrunch

How can such a small team achieve the grand goal of general artificial intelligence?

In the current era where Scaling Laws have not yet lost their efficacy, on one hand, companies using large models as the cornerstone of AGI continue to expand their parameter and data scales. On the other hand, Apple represents the trend of focusing on efficiency rather than parameters for small models on the client side. However, the groupthink among different research teams at major companies, repeating the same work over months, baffles Carmack.

In Carmack's view, research in the field of machine learning has a degree of randomness, and there is no clear direction yet. The advantage of small teams is that they can see themselves as random test points, starting research from random weights and positions, thereby having a higher probability of finding the right path.

Amid the dominance of the Transformer in large model architectures, new architectures are attempting to challenge its supremacy.

For example, the Mamba neural network architecture, released by two teaching assistants from Carnegie Mellon University and Princeton University, garnered heated responses in the open-source community upon its December release last year.

According to Tri Dao, an assistant professor at Princeton and co-author of the architecture, Mamba breaks through the limitation of the Transformer's core attention layer, which cannot scale to long contexts. Mamba can linearly scale with the increase in context length, offering five times the inference throughput, and the language model based on this architecture outperforms Transformer-based models.

Recently, the Mamba-2 architecture, led by the original author, was officially released, showing significant improvements in training speed and task execution.

Turning to Keen, Casson stated that he would focus on the company's long-term development and not rush into commercialization. Currently, the Keen team is exploring enhancements in the actions, perception, and cognitive abilities of intelligent agents, while also seeking precise metrics to evaluate AI progress.

Over the past two years, the company has not sought continuous funding, which is almost unimaginable in the cash-burning AI startup scene.

In an interview, Casson mentioned that AGI is a ten-year mission, and he does not want to burn through $20 million in two years only to seek another series of funding for several hundred million more.

He plans to spend several years addressing some of the challenges of AGI, extending the results achieved to a certain extent, and observing their effects. Once the intelligent agents he has developed become capable of learning and interaction, Casson will form a large, diverse team to conduct further research.

Currently, the path to AGI is still shrouded in mystery.

However, just as he has made his mark in the fields of 3D gaming, rockets, and VR through vision and solid execution, Casson is sure to leave an impressive legacy in this new, unexplored territory.