Notes on gen ai in production
Posts are grouped into topics, each one a thread you can read straight through. Pick a topic in the left column or search by keyword.
The JEV Ecosystem: Official SDKs, Community Projects and Open System One Models
A compact list of the tools and repositories in the JEV ecosystem, grouped into three layers by who is responsible: official (the typesafe-ai org), community (calling the real JEV through the SDK or HTTP), and open / self-hosted (open models behind a server that speaks the same protocol). One line per tool: name, link, description.
Decision Models: A New Branch Beside Generative LLMs
Most places we wire an LLM into software only need a decision: which team gets this ticket, should this transaction be blocked, does a person need to look. Yet we still make the model write prose and parse it back. A decision model flips that: declare the answer space first, get back a typed decision with probabilities.
Datasets for LLMs: From Raw Data to the Training Samples a Model Actually Reads
A web page is not a training sample. Neither is a book. This post walks the whole distance in between - collection, extraction, normalization, filtering, dedup, decontamination, domain mixing, tokenization, chunking - to show why the dataset is not side work outside the model, but part of the very problem of building an LLM.
Fine-tuning a Pretrained Model: From Raw Data to the Training Loop
The first two parts were about a model that already knows things. This one is about changing what it knows: splits and data leakage, padding and the real cost of a tensor, cross-entropy, backpropagation, AdamW, learning-rate schedules, and reading a learning curve to tell overfitting from underfitting.
Tokenization in LLMs: From Text Humans Can Read to Numbers a Model Can Process
The first three parts all started from a point where text had already become numbers. This one goes back to that step: normalization, pre-tokenization, subword segmentation, vocabulary and special tokens - the discrete interface that decides your context window, your costs, and even whether a language pays a token tax.
Using Transformers: From pipeline() to Optimized LLM Deployment
pipeline() is only a thin layer of abstraction. This post opens it up: the tokenizer, the model head, logits, padding and attention masks, then on to device, dtype, the KV cache, continuous batching, quantization, and how to choose between vLLM, SGLang and llama.cpp now that TGI is archived.
Transformer from A to Z: Understanding the Foundation of LLMs
A systematic mental model of the Transformer: from tokens and why some languages cost more of them, through Attention, residuals, the FFN and RoPE, to the three architecture families, MoE and the full path of inference.
RAG will not save dirty data
Three months debugging an internal QA system, and 80% of the work turned out to be document normalization.
No posts match the current filters.
Once a month, new posts only. No ads, unsubscribe anytime.