Skip to content

November 8, 2023

Google Leaked Memo “We Have No Moat (and Neither Does OpenAI)” through the Lens of Slowify, Simplify, Amplify

By Gene Kim

Yesterday was the first OpenAI DevDay, which so many of us watched with excitement and amazement.

This reminded me of the May 2023 leaked “We Have No Moat, And Neither Does OpenAI” memo from a Google researcher. It was an incredibly fascinating memo that argued against the notion that frontier LLMs (Large Language Models), trained on massive amounts of text data at a cost of tens or even hundreds of millions of dollars, will be the winning solution.

The anonymous researcher goes even further, making the claims that these huge, capital-intensive projects don’t even create a moat. (Indeed, these frontier LLMs are very capital-intensive, requiring massive amounts of money and time. And they tie up the already incredibly-scarce GPU capacity — check out today’s article on how Microsoft’s need for GPUs are so high, they’re renting them from Oracle Cloud!)

What I find so tantalizing and startling about the memo: one of the main themes seemed very familiar. One of the things we’ve learned is that modularity in software is incredibly desirable — it enables independence of action, which enables vast numbers of people to work independently of each other.

In my soon-to-be-released book, Wiring the Winning Organization, co-authored with Dr. Steve Spear, we have case studies of the great Amazon re-architecture of 2001 and the massive $5 billion investment that IBM made in the 1960s to create the System/360 project (that’s $20 billion in today’s dollars!). What these two case studies have in common is that they created an architecture that allowed for teams to work in parallel, enabling teams to work and innovate independently.

What’s frankly amazing and mind-blowing in the memo are the observations about the pace of development happening in the OSS AI model community. To pick a few:

  • After the release of Meta’s LLaMA: “A tremendous outpouring of innovation followed, with just days between major developments (see The Timeline for the full breakdown). Here we are, barely a month later, and there are variants with instruction tuning, quantization, quality improvements, human evals, multimodality, RLHF, etc., many of which build on each other.”
  • “By contrast, training giant models from scratch not only throws away the pretraining but also any iterative improvements that have been made on top. In the open source world, it doesn’t take long before these improvements dominate, making a full retrain extremely costly.”

This is a fantastic document that anyone interested in AI should read. It’s interesting to see all the amazing advancements that OpenAI announced yesterday, and I’m not smart enough to even opine on to what extent this refutes or validates the claims in the paper.

But I find the arguments very, very compelling.

As I’ve done with the Cloudflare outage post-mortem and the Nordstrom case studies, I had ChatGPT interpret this memo through the lens of Slowify, Simplify, and Amplify.

Here’s what it came up with — edited by me to improve clarity and further land the point.

Slowification:

  • This memo proposes a reassessment of Google’s approach to AI development by proposing a thoughtful and strategic response to the changing landscape, specifically around OSS AI models.
  • The memo suggests that there is much to learn from the open-source community, pointing out the incredible pace of innovation. It provides a stunning timeline of advancements in the 50 days after the leak of the LLaMA source code (and the weights being leaked one week later)
  • The memo describes the shortcomings of direct competition with open-source initiatives and proposes a strategic pivot to create an ecosystem that helps open-source efforts.

Simplification:

  • Incrementalization: The ability for smaller AI models to soon run on consumer hardware (e.g., laptops, phones, glasses) will enable much smaller, incremental steps than currently possible with frontier LLMs.
  • Modularization:
  • The pace of innovations happening in open-source AI efforts is astonishingly fast. As an example, the creation of Low Rank Adaptation (LoRA) was cited as just one new mechanism that helps quickly and efficiently update models, enabling rapid and distributed innovation.
  • Smaller models, models working together, and complementary efforts allow for the separate enhancement and sharing of model components, encouraging parallel development efforts. In other words, more brains beat fewer brains almost all of the time.
  • Open source provides a decentralized model of innovation where the collective intelligence of the open-source community can contribute to the evolution of AI models.

Amplification:

  • The memo itself is an example of an attempt to amplify a signal that Google might be at a competitive disadvantage compared to open-source models, signaling a need for urgent attention and action.
  • By discussing the successes of the open-source community, the memo proposes the need for Google to adapt its strategy to incorporate more open and collaborative approaches.
  • The memo pushes the organization to consider and act upon the challenges it faces more promptly and effectively before it is too late.

I find the logic of this argument to be rock-solid. How this squares with OpenAI’s amazing pace of innovation and shipping capabilities that developers can use, I’m not quite sure.

What do think?

(Also, many of you were interested in the prompts I used. I will be publishing the prompts on GitHub. Please let me know if you find them of value; feel free to submit suggestions/PRs, etc.: https://github.com/realgenekim/wwo-llm-prompts/blob/main/prompts/wwo-simple-prompt.md)

- About The Authors
Avatar photo

Gene Kim

Gene Kim has been studying high-performing technology organizations since 1999. He was the founder and CTO of Tripwire, Inc., an enterprise security software company, where he served for 13 years. His books have sold over 1 million copies—he is the WSJ bestselling author of Wiring the Winning Organization, The Unicorn Project, and co-author of The Phoenix Project, The DevOps Handbook, and the Shingo Publication Award-winning Accelerate. Since 2014, he has been the organizer of DevOps Enterprise Summit (now Enterprise Technology Leadership Summit), studying the technology transformations of large, complex organizations.

Follow Gene on Social Media

1 Comment

  • Anonymous Nov 12, 2023 4:42 pm

    Thank you for the in-depth explanation. I also learned that modularity in software is highly desirable, but few communities understand it. Here are my principles of modularity in my daily work on developing a modular source code network.1. Environment: 1.1. management over creation 1.2. requirements over expectations2. Source-code: 2.1 reusability over code development 2.2 writing over refactoring code3. Data: 3.1 data standardization first 3.2 fat-data over fat-functionWhat do you think about Hyper modular architecture?

    Reply

Leave a Comment

Your email address will not be published.



Avatar photo

Gene Kim

Jump to Section

    More Like This

    The Missing Link in Your Industry 4.0 Strategy: Industrial DevOps
    By Summary by IT Revolution

    As manufacturers embrace Industry 4.0, many find that implementing new technologies isn't enough to…

    Read Article
    The Original Disruptor of the Music Industry
    By Matt McLarty , Stephen Fishman

    I know. You’re thinking I'm talking about Napster, right? Nope. Napster was launched in…

    Read Article
    From Turbulence to Transformation: A CIO’s Journey at Southwest Airlines
    By Summary by IT Revolution

    When Southwest Airlines' crew scheduling system became overwhelmed during the 2022 holiday season, the…

    Read Article
    High Stakes Communication: The Four Pillars of Effective Leadership Communication
    By Summary by IT Revolution

    You've been there before: standing in front of your team, announcing a major technological…

    Read Article