Knowledge Pack · AI & tech

LLM 101

A clear primer on large language models: their history from n-grams to the Transformer to ChatGPT, how they work (tokens, attention, context windows), the training pipeline (pretraining, SFT, RLHF, alignment), prompting and RAG, the key companies and people, and the numbers and milestones that define the field.

604 cards10 lessonsFree

Get this pack on your phone

Scan with your phone's camera.

flashfeedlearningapp.com/packs/llm-101

Part of the Learn AI collection.

Constitutional AI's self-critiqueAI-generated (gpt-image-1.5)Pre-Transformer NLP: n-gramsUnsplash · Glen Carrie · Unsplash License ↗Why hallucinations cost youAI-generated (gpt-image-1.5)
What's inside

10 lessons.

Each lesson has a study guide to read and objectives you can test yourself on. Its cards and games show up in your feed. Try one of its cards right here. Play a few of its games here too.

  1. Pre-Transformer NLP: n-gramsUnsplash · Glen Carrie · Unsplash License ↗
    01

    A Brief History of LLMs

    Before the Transformer · The GPT era · ChatGPT and the modern wave

  2. Learned patterns produce two answer sheets that look exactly alike, one marked Correct with a check and one Wrong with a cross: both sound equally plausible.FlashFeed original diagram · CC-BY-4.0
    02

    How LLMs Work

    Tokens, embeddings and attention · How text is generated

  3. Constitutional AI's self-critiqueAI-generated (gpt-image-1.5)
    03

    The Training Pipeline

    Pretraining to RLHF · Alignment, inference and compute

  4. Prompting vs. RAG vs. fine-tuningPexels · Diana ✨ ↗
    04

    Prompting, RAG and Using LLMs

    Prompting techniques · RAG, tools and agents

  5. 05

    Companies, People and Models

    The companies · The godfathers and key researchers · Founders, leaders and the ecosystem

  6. Take the lessons with you. Free, no account.Download on the App Store ↗Get it on Google Play ↗
  7. 06

    Key Numbers and Milestones

    Model sizes and dates · Scaling, context and misconceptions

  8. A developer writing code on a laptopPexels · cottonbro studio · Pexels License ↗
    07

    What LLMs Are Good For

    Language and writing tasks · Knowledge, search and code · Work and everyday tasks · High-stakes domains and choosing well

  9. Why hallucinations cost youAI-generated (gpt-image-1.5)
    08

    Limits and Responsible Use

    Reliability and staleness · Bias, privacy and your data · Security and societal risks · Limits, safeguards and deployment

  10. 09

    Quick Recall

    Core terms

  11. 10

    Key Concepts

    Key People · Key Things

What you'll be able to do 47 objectives, from the pack's lessons
  • Trace the pre-Transformer progression from RNNs/LSTMs through the attention mechanism to the 2017 Transformer.
  • Contrast the Kaplan (2020) and Chinchilla (2022) scaling laws and what each revised about optimal model training.
  • Explain how BERT's bidirectional (masked) pretraining differed from GPT and identify the GPT-1 to GPT-3 progression.
  • Describe how RLHF/InstructGPT and the ChatGPT and GPT-4 launches turned capable models into useful products.
  • Recognize the modern open-weight wave (LLaMA, Mistral) and the frontier assistants (Claude, Gemini, GPT-4).
  • Explain how text becomes tokens and then embedding vectors a model can compute over.
  • Describe the Transformer, self-attention, context window, and parameters that make up a model's internals.
  • Describe next-token prediction and autoregressive generation as the core loop that produces text.
  • Explain how logits and the temperature knob shape which next token is sampled.
  • Explain why LLMs hallucinate and recognize emergent abilities that appear only at scale.
  • Explain self-supervised pretraining and why the resulting base model is not yet a helpful assistant.
  • Describe supervised fine-tuning as training on prompt/response pairs to make a model follow instructions.
  • Explain how RLHF and its reward model align a model, and how RLAIF and Constitutional AI scale that alignment.
  • Identify the instruct/chat model as the aligned product of the full base-plus-SFT-plus-RLHF pipeline.
  • Distinguish training from inference and relate the FLOPs compute cost to why training a large model is costly but querying it is cheap.
  • Describe what a prompt and a system prompt contain, and why prompting, though free, can't give a model knowledge it wasn't trained on.
  • Distinguish zero-shot, few-shot and chain-of-thought prompting, explain in-context learning, and recognize step-by-step reasoning as emergent.
  • Explain RAG and its supporting components — vector databases and chunking — for grounding a model in external knowledge.
  • Describe function calling and LLM agents, and how an agent's planning loop differs from a single-prompt interaction.
  • Contrast fine-tuning with RAG as ways to add knowledge or behavior, one baking into weights and the other into context.
  • Match the US frontier labs (OpenAI, Anthropic, Meta, xAI) to their flagship model families.
  • Identify the international and second-tier labs (Google DeepMind, Mistral, Cohere) and the lab that drove open-source LLMs.
  • Recognize the three 'godfathers of deep learning' who shared the 2018 Turing Award.
  • Identify pivotal LLM researchers — Vaswani, Sutskever, and Karpathy — and their contributions.
  • Match today's lab leaders (Altman, the Amodeis, Hassabis) and the infrastructure players (Nvidia, Hugging Face).
  • Recall the parameter counts of the GPT line (GPT-1, GPT-2, GPT-3) and GPT-3's training-data size.
  • Recall the parameter counts of other landmark models (BERT-large, LLaMA's 7B-to-65B family, Mistral 7B and PaLM) and compare them with GPT-3.
  • Place the key release years in order: BERT, GPT-2, GPT-3, LLaMA, ChatGPT, and GPT-4.
  • Recall the ChatGPT and GPT-4 launch years and ChatGPT's record two-month climb to 100 million users.
  • Recall the scaling-law figures (Kaplan year, Chinchilla's ~20 tokens/parameter) and context-window growth, and why bigger isn't always better.
  • Identify LLM strengths across language tasks — summarization, translation, rewriting, editing, classification, and simplification.
  • Explain LLM knowledge/search use cases (research, semantic search, extraction, RAG) and coding assistance (Copilot speedup, explanation, debugging, tests).
  • Identify everyday work and personal tasks LLMs assist — support, drafting, meeting notes, data cleaning, tutoring, brainstorming — with their catches.
  • Apply the copilot pattern and the cost-of-verification framework to decide how much automation a task warrants.
  • Judge good vs poor LLM fit and the human-review guardrails required in healthcare, legal, and finance.
  • Explain why LLMs hallucinate and what a knowledge cutoff means for stale, confidently-stated answers.
  • Identify where LLM bias enters (data collection, pretraining, fine-tuning) and how it shows up, such as the AAVE accuracy gap.
  • Describe what can happen to data pasted into a consumer LLM and the memorization/PII-extraction risk.
  • Explain prompt injection and jailbreaks, automation bias, and why tool-enabled agents widen the attack surface.
  • Recognize red-teaming and model cards, explain how non-determinism limits reproducibility, and why filters alone don't make a model safe for high stakes.
  • Recall the core architecture terms — Transformer, token, self-attention, and context window — instantly.
  • Recall the training terms — pretraining, RLHF, and Chinchilla's ~20-tokens-per-parameter rule.
  • Recall the usage terms — RAG, inference, hallucination, few-shot, and emergence.
  • Recognize the three Turing-Award 'godfathers of deep learning' — Bengio, Hinton, and LeCun.
  • Identify key researchers behind modern LLMs — Sutskever and Radford at OpenAI, and educator Andrew Ng.
  • Recognize foundational techniques and architectures — word2vec, LSTM, and the Transformer.
  • Recognize landmark models — BERT, GPT-3, and ChatGPT — including GPT-3's 175-billion-parameter scale.
Free to read

Study guide.

The same study guide as in the app, one page for each lesson, with its key facts and a self-test.

  1. 01A Brief History of LLMs
  2. 02How LLMs Work
  3. 03The Training Pipeline
  4. 04Prompting, RAG and Using LLMs
  5. 05Companies, People and Models
  6. 06Key Numbers and Milestones
  7. 07What LLMs Are Good For
  8. 08Limits and Responsible Use
  9. 09Quick Recall
  10. 10Key Concepts
Where this comes from

Built from 2 sources.

4 of its 606 cards cite a source. These are the sources it cites.

  1. Peng et al. 2023, The Impact of AI on Developer Productivity: Evidence from GitHub Copilot ↗arxiv.org · cited on 3 cards
  2. Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks ↗arxiv.org · cited on 1 card
Photo credits

Hover over a photo, or tap its i, to see where it came from. All 6:

About this pack
Cards
604
Lessons
10
Version
1.1.21
Language
English

The whole pack is one file. Read the pack file ↗

Now on iPhone and Android

FlashFeed is on the App Store and Google Play.

FlashFeed is free on iPhone and Android, and needs no account.

More in AI & tech
All packs →