LLM & NLP

How language models work: tokens, attention, sampling, embeddings, RAG, hallucinations. · 11 articles

All Claude Code Claude API LLM & NLP AI Agents & Skills Tips AI Automation Tools System Design Reading

LLM & NLP3 min read

Scaling laws and Chinchilla: why small models keep getting stronger

From Kaplan's 2020 power laws to Chinchilla's ~20 tokens per parameter, and why models are now over-trained to be cheap to serve.

LLM & NLP3 min read

RLHF and Constitutional AI: how raw models become helpful assistants

InstructGPT's three RLHF steps, how Constitutional AI uses principles and AI feedback, a comparison of the two, and side effects like sycophancy and verbosity.

LLM & NLP2 min read

From chain-of-thought to reasoning models: giving LLMs scratch paper

Three foundational studies (CoT, step-by-step, self-consistency), why step-by-step reasoning works, and using it with Claude's thinking.

LLM & NLP2 min read

Mixture of Experts (MoE): huge models where each token uses only a fraction

Routers and experts, three milestones (Sparsely-Gated MoE, Switch Transformer, Mixtral), and understanding total vs active parameters when self-hosting.

LLM & NLP3 min read

What is a token? BPE and why some languages cost more

Tokenization and Byte Pair Encoding explained: why models miscount letters, why non-English text costs more tokens, and how to count tokens with the Claude API.

LLM & NLP3 min read

Transformers and self-attention: the foundation of every LLM

Self-attention, Query/Key/Value and multi-head explained, plus why the Transformer makes long context expensive and caching effective.

LLM & NLP3 min read

Temperature, top-p, top-k: how an LLM picks the next word

How sampling works in LLMs, what temperature, top-p and top-k do, how to set them on the Claude API per task, and common misconceptions.

LLM & NLP3 min read

Embeddings and semantic search: from word2vec to Voyage

What embeddings are, why meaning-based search beats keywords, using them with Claude via Voyage AI, input_type, quantization and common traps.

LLM & NLP3 min read

Contextual Retrieval: fixing RAG's root problem

Anthropic's Contextual Retrieval: prepend context to each chunk and combine embeddings, BM25 and reranking to cut retrieval failures by 67%.

LLM & NLP3 min read

Why LLMs hallucinate and 7 ways to reduce it

The causes of hallucination per 2025 research (training plus scoring that rewards guessing), common types, and Anthropic's mitigation techniques.

LLM & NLP3 min read

Context rot and "lost in the middle": longer context isn't always better

Two studies show LLMs use context unevenly: information in the middle gets missed and quality drops as input grows. Plus six mitigation tactics.

No matching articles

Try another keyword or clear the topic filter.

LLM & NLP

Never miss a new article