Deployment map: TGI, vLLM and llama.cpp

Deployment map: TGI, vLLM and llama.cpp An architecture diagram generated by Archify. Application · chat / RAG / agent · Architecture component Application chat / RAG / agent OpenAI-compat API · /v1/chat · Architecture component · easy backend swap OpenAI-compat API /v1/chat easy backend swap TGI · HF serving stack · Production GPU serving · streaming + metrics TGI HF serving stack streaming + metrics vLLM · high throughput · Production GPU serving · many users vLLM high throughput many users llama.cpp · GGUF quantized · Local inference · CPU-first, offline llama.cpp GGUF quantized CPU-first, offline KV cache · PagedAttention · Production GPU serving · less fragmentation KV cache PagedAttention less fragmentation FlashAttention · block-wise kernel · Production GPU serving · fewer HBM reads/writes FlashAttention block-wise kernel fewer HBM reads/writes Datacenter GPU · A100 / H100 · Production GPU serving Datacenter GPU A100 / H100 Personal machine · CPU + discrete GPU · Local inference · offload layer Personal machine CPU + discrete GPU offload layer one API standard HF-centric throughput offline batching optimized kernel uses VRAM fewer HBM reads Q4_K_M Production GPU serving Local inference Legend Frontend Backend Database Cloud External