Tensor Core
GPU Hardware & ComputeSpecialized processing units within NVIDIA GPUs designed to accelerate the matrix multiplications at the heart of neural network training and inference.
The B300 GPUs in Eleveight AI's cluster are packed with tensor cores supporting FP8, FP16, and BF16 precision, optimized for both the training and inference phases of the AI lifecycle.
Overview
Tensor cores are a class of processing unit NVIDIA introduced in 2017 and has refined in every GPU generation since. The distinction from ordinary cores is one of granularity: where a standard CUDA core performs a single floating-point operation per clock cycle, a tensor core executes an entire matrix multiply-accumulate operation in one, doing in a single step what would otherwise take many.
How it works
A tensor core takes two small matrices, multiplies them together, and adds the result to a third, all as one fused operation. Because the layers of a neural network are, at bottom, long sequences of matrix multiplications, tensor cores map directly onto the exact computation that dominates AI workloads, turning the most common operation into the most efficient one.
Why it matters
The B300's tensor cores support FP8 arithmetic, a lower-precision number format that preserves model quality while roughly doubling throughput compared with FP16. Lower precision means more operations completed in the same time and memory, and that efficiency is a large part of why the B300 represents such a marked step forward, particularly for inference performance.
Use cases
- Accelerating matrix multiplications in neural network training
- High-throughput inference using FP8 precision
- Mixed-precision training for memory and compute efficiency