Transformer trong một lượt inference

Transformer trong một lượt inference An architecture diagram generated by Archify. Prompt · user + context · Architecture component Prompt user + context Tokenizer · text -> token ids · Architecture component Tokenizer text -> token ids Embedding · token + position · Decoder-only LLM inference path Embedding token + position Transformer block · Q/K/V + FFN · Decoder-only LLM inference path · core Transformer block Q/K/V + FFN core Logits · scores over vocab · Decoder-only LLM inference path Logits scores over vocab Sampling · temperature / top-p · Decoder-only LLM inference path Sampling temperature / top-p Next token · append + repeat · Decoder-only LLM inference path Next token append + repeat KV Cache · past keys / values · Decoder-only LLM inference path · decode KV Cache past keys / values decode text ids vectors hidden state distribution reuse past K/V Decoder-only LLM inference path Legend Backend Database Security Message bus External