Parameter
AI & Machine LearningOne of the adjustable values inside a model that training tunes; the count of them, often in the billions, is a rough measure of a model's scale and capability.
When a model's parameter count runs to billions, the memory and bandwidth needed to train and serve it grow accordingly, which is exactly what the HBM3e capacity of Eleveight AI's B300 GPUs is built to supply.
Overview
A parameter is a single tunable number within a model, and the full collection of them is what the model adjusts as it learns. Parameter count has become the headline figure for model scale, a seven-billion-parameter model, a seventy-billion-parameter model, because it correlates strongly, though not perfectly, with capability. More parameters give a model more capacity to represent complex patterns, at a proportional cost in the compute and memory required to train and run it.
How it works
Parameters are chiefly the weights on the connections between a network's units, plus related values such as biases. They begin essentially random and are refined throughout training as the model learns from data. At inference time, every parameter must be held in memory and participate in the calculation for each request, which is why a model's parameter count translates so directly into its hardware footprint.
Why it matters
Parameter count sets concrete hardware requirements. A model's weights must fit in GPU memory, so larger models demand more of it, frequently more than a single GPU can offer, which forces the work to be split across several. The relationship between scale and capability is much of what has driven demand for ever-larger memory and ever-faster interconnects, and it is the reason memory capacity is a first-order constraint in AI infrastructure.
Use cases
- Sizing hardware for a given model
- Planning multi-GPU deployment for large models
- Comparing model scale and capability
- Estimating training and inference cost