Foundation Model

Overview

Foundation models, the family that includes GPT-4, Claude, Llama, and Gemini, are trained on enormous, broad datasets at considerable scale. The insight behind them is economic as much as technical: rather than building a separate specialized model for every task, one general model learns rich, reusable representations that can then be steered toward many different applications with comparatively little extra effort.

How it works

Their creation typically unfolds in two stages. First comes pre-training, in which the model absorbs a vast corpus and develops broad internal representations of language, images, or code. Then comes adaptation, through fine-tuning on smaller curated datasets or through alignment techniques, which shapes that general capability toward specific behaviors, domains, or tasks without starting over.

Why it matters

Foundation models dramatically lower the barrier to building useful AI. An organization can begin from an already-capable model and adapt it, rather than shouldering the immense cost of training something from nothing. That shift is largely what moved advanced AI out of a handful of well-funded labs and into the reach of ordinary product teams.