What an LLM system is made of

What an LLM system is made of An architecture diagram generated by Archify. Text · your prompt · Architecture component Text your prompt Tokenizer · vocab + template · Architecture component · wrong → garbage output Tokenizer vocab + template wrong → garbage output Architecture · layer, head, hidden · Model artifact — loaded from one checkpoint · decides shape Architecture layer, head, hidden decides shape Weights · safetensors · Model artifact — loaded from one checkpoint · decides quality Weights safetensors decides quality Task head · classification / LM · Model artifact — loaded from one checkpoint · decides output Task head classification / LM decides output Generation · temperature, top-p · Runtime — you configure · wrong → loops or dull text Generation temperature, top-p wrong → loops or dull text Memory · KV cache, dtype · Runtime — you configure · wrong → out of VRAM Memory KV cache, dtype wrong → out of VRAM Inference engine · batching, streaming · Runtime — you configure · decides throughput Inference engine batching, streaming decides throughput Hardware · CPU / GPU / VRAM · Architecture component · hard ceiling Hardware CPU / GPU / VRAM hard ceiling character string ids + mask load parameters hidden states logits cached K, V chosen token kernels + scheduling Model artifact — loaded from one checkpoint Runtime — you configure Legend Frontend Backend Database Cloud Security Message bus External