The Shift from Models to Compound AI Systems – The Berkeley Artificial Intelligence Research Blog

By neub9
3 Min Read

In 2023, AI took the world by storm with the introduction of Large Language Models (LLMs) capable of performing general tasks like translation and coding through simple prompts. As developers began utilizing LLMs to build AI applications, a shift became evident. Researchers and companies increasingly turned towards compound AI systems with multiple integrated components, as opposed to relying solely on monolithic models.

For instance, Google’s AlphaCode 2 and AlphaGeometry achieved state-of-the-art results in programming and Olympiad problems by engineering systems that combined LLMs with traditional solvers. Meanwhile, Databricks found that many LLM applications adopted retrieval-augmented generation (RAG) and multi-step chains. Even researchers previously focused on single LLM calls have transitioned to complex inference strategies to improve performance.

This transition to compound systems raises fascinating design questions and highlights the importance of engineering ingenuity in AI advancement, rather than just scaling up training. In this post, we delve into the shift towards compound AI systems, the implications for developers, and the emerging tools for developing and optimizing these systems, which have received less attention than model training.

We argue that compound AI systems will likely dominate AI development in the future and may be one of the most impactful trends in AI in 2024. Despite continuous improvements in AI models, recent state-of-the-art results increasingly stem from compound systems. This is driven by several factors, including the ability to improve performance via system design, the dynamic nature of systems, the ease of improving control and trust, and the capability to tailor performance to specific needs. Additionally, the shift to compound systems aligns with trends in other AI fields, such as self-driving cars.

While compound AI systems offer clear benefits, designing, optimizing, and operating them remains an evolving art. There are many design questions to address, such as whether the control logic should be driven by traditional code or AI models, determining where to invest resources in a compound system, and optimizing compound systems end-to-end to maximize performance.

To illustrate the breadth of design choices in compound AI systems, we provide examples such as AlphaCode 2, AlphaGeometry, Medprompt, Gemini on MMLU, and ChatGPT Plus. These systems showcase the potential and versatility of compound AI systems, each with its unique components and design.

Overall, as AI models continue to evolve, compound AI systems are expected to play a crucial role in maximizing AI results. The design, optimization, and operation of compound systems will be areas of high importance for AI developers in the coming years.

Share This Article
Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *