Navigating the AI WorldPart D · Models, platforms and hardwareChapter 25 of 28

Read the seven-layer map correctly

The seven-layer picture is a dependency map, not the path every prompt travels through. Training usually happens before deployment. At runtime, inference uses the resulting model together with the context and tools supplied by the application.

Figure 7. A dependency map from applications to physical facilities. Training creates and adapts model artifacts before deployment, while runtime services execute deployed models with permitted context and tools.
Figure 7. A dependency map from applications to physical facilities. Training creates and adapts model artifacts before deployment, while runtime services execute deployed models with permitted context and tools.

Applications determine the user experience and work process. Models provide learned capability. Data and training shape that capability and produce the artifacts that will be evaluated and deployed. Serving infrastructure loads models, schedules requests and connects storage and networks.

Accelerators perform the high-throughput numerical work. Semiconductors, memory and packaging determine how much computation and data movement the hardware can sustain. Energy, cooling and facilities keep that hardware operating. Networking crosses several layers, so it is more useful to treat it as a shared dependency than to force it into a single box.

The map helps connect software decisions to physical consequences. Longer requests consume more computation and working memory. Larger models can require more devices and more communication. Higher utilization can reduce unit cost while increasing contention. Deployment geography affects service design as well as physical constraints.

Most professionals only need enough of this map to understand those consequences. A larger context window can change latency and cost. A larger model can change device count and communication. Go deeper when those tradeoffs become your responsibility.

Report a correction

Corrections go to the editor and are never published automatically. No account needed.