Small Model Approach

Full Title or Meme

Training a Large Language Model is very expensive and consumes a great deal of memory. For Artificial Intelligence to become available to all people without exposing their queries to the cloud, a Small Model Approach to AI is required.

Apple

1. **Apple Intelligence**:

  - Apple recently introduced **Apple Intelligence**, a personal intelligence system integrated into iOS 18, iPadOS 18, and macOS Sequoia.
  - It harnesses the power of Apple silicon to understand and create language and images, take action across apps, and draw from personal context to simplify and accelerate everyday tasks³.
  - Apple Intelligence is deeply integrated into the operating systems, seamlessly blending generative AI aspects without requiring users to understand the underlying technologies.
  - The focus is on a frictionless user experience, aligning with Apple's design philosophy.

2. **Small-Model Approach**:

  - Unlike other companies that prioritize large models, Apple takes a more pragmatic approach.
  - Apple Intelligence uses smaller, topic-specific models trained on customized datasets for specific functionality.
  - These smaller models increase transparency around decision-making and allow users to report issues with the generative AI system.
  - While more focused, Apple Intelligence can still cover a spectrum of requests thanks to specialized "adapters" for different tasks and styles¹.

Source: Conversation with Copilot, 7/10/2024

(1) Introducing Apple Intelligence for iPhone, iPad, and Mac. https://www.apple.com/newsroom/2024/06/introducing-apple-intelligence-for-iphone-ipad-and-mac/.
(2) Why Apple is taking a small-model approach to generative AI. https://techcrunch.com/2024/06/11/why-apple-is-taking-a-small-model-approach-to-generative-ai/.
(3) Apple's AI Breakthroughs: Vision, Multimodal AI, and On-Device Agents. https://summaries.getnotable.ai/apples-ai-breakthroughs-vision-multimodal-ai-and-on-device-agents/.
(4) What is Apple Intelligence? - 9to5Mac. https://9to5mac.com/2024/06/11/what-is-apple-intelligence-heres-how-ai-will-change-the-way-you-use-your-iphone/.

References

• See wiki page on MobileLLM for an approach targeting Smartphones.