Databases & Data — where your application's data lives, from a first table to production

Almost every program you build eventually needs to remember something — a user, a setting, a log of decoded calls. A database is where that data lives: structured so you can store it reliably and get it back fast. This module starts from why plain files aren't enough, builds up the relational model and SQL — the language nearly every database speaks — then teaches you to design data well, reach beyond relational to key-value, document, time-series, and vector stores, and finally talk to a database from your own code and run one in production. Examples lean on real software, including how a project like GopherTrunk stores the calls and systems it decodes. Pairs with the Software Engineering and AI Software Development paths; no prior database experience assumed.

Start lesson 1 →Jump to glossary

Almost every program you build eventually needs to remember something — a user, a setting, a log of the calls it decoded. A database is where that data lives: structured so you can store it reliably and get it back fast, even with many readers and writers at once. Learning databases is one of the highest-leverage things a developer can do, because the same handful of ideas — tables, keys, SQL, transactions — show up in almost every application ever built.

Who this is for. Anyone who can write a little code (or wants to) and keeps bumping into the question of where does the data go? You don’t need any database background — this starts from why plain files aren’t enough and builds up from there. It pairs naturally with the Intro to Software Dev, Programming in Go, and Building AI Into Software modules.

How the module works. Six units take you from a first table to a database running in production. The early units build the core ideas — the relational model, SQL, and how to design data well. The middle unit reaches beyond relational to the NoSQL, time-series, and vector stores that modern apps lean on, including the vector search behind AI retrieval. The last two units are the working developer’s part: talking to a database from your own code (safely, in Go), and the operational reality of backups, scaling, and monitoring. Examples lean on real software, including how a scanner like GopherTrunk stores the systems and calls it decodes. Mark lessons complete as you go — your progress is saved in your browser. New here? Start with lesson 1: What a database is

Unit 1 — Why Data Needs Structure

What a database is and the ideas underneath it: persistence, the relational model, schemas and types, and the keys that tie data together.

  1. What a database is (and why not just files) The problem databases solve — storing data so many readers and writers can query it reliably, safely, and fast, instead of hand-rolling files. beginner 8 min
  2. Data, state & persistence The difference between data your program holds in memory and data that outlives it — persistence, durability, and why "save it somewhere" is harder than it looks. beginner 8 min
  3. The relational model — tables, rows & columns The idea that has run the data world for fifty years — organising data into tables of rows and columns, and why that simple shape is so powerful. beginner 9 min
  4. Schemas, columns & data types A schema is the shape of your data — the columns, their types, and the rules — and why deciding it up front saves you from a mess later. beginner 8 min
  5. Keys & relationships Primary keys that name each row and foreign keys that link tables — how a database models the relationships between your things. beginner 9 min

Unit 2 — SQL, the Language of Data

The query language almost every database understands: selecting, filtering, joining, aggregating, changing data, and making it all fast with indexes.

  1. What SQL is The declarative language for talking to a relational database — you describe the data you want, not how to fetch it, and the database works out the rest. beginner 8 min
  2. Querying with SELECT The single most-used statement in all of software — SELECT — and how to ask a database for exactly the columns and rows you need. beginner 9 min
  3. Filtering, sorting & limiting WHERE, ORDER BY, and LIMIT — narrowing a result set to the rows that matter, in the order you want, without dragging back the whole table. beginner 9 min
  4. Joining tables The heart of relational power — combining rows from two tables on a shared key, and the difference between inner and outer joins. intermediate 10 min
  5. Aggregation & GROUP BY Turning many rows into a summary — counts, sums, and averages — with GROUP BY and HAVING, the tools behind every dashboard number. intermediate 9 min
  6. Inserting, updating & deleting The write side of SQL — INSERT, UPDATE, and DELETE — and the WHERE clause you forget at your peril. beginner 8 min
  7. Indexes & how queries get fast Why the same query can take a millisecond or a minute — indexes, what they cost, and the mental model of a book's index that explains them. intermediate 10 min

Unit 3 — Designing Data Well

Turning a pile of columns into a sound design: normalization, modeling a real app, constraints, transactions, and evolving the schema safely over time.

  1. Normalization & avoiding duplication Why storing the same fact in two places will eventually betray you — normalization, and the handful of normal forms worth actually knowing. intermediate 10 min
  2. Data modeling for a real app Going from "what does my app do" to a set of tables and relationships — the practical craft of modeling data before you write a line of code. intermediate 10 min
  3. Constraints & data integrity Letting the database enforce the rules — NOT NULL, UNIQUE, CHECK, and foreign-key constraints that keep bad data out no matter what the app does. intermediate 9 min
  4. Transactions & ACID All-or-nothing changes — transactions, the ACID guarantees behind them, and why moving money (or updating two tables) needs them. intermediate 10 min
  5. Schema migrations & evolving a database Your schema is never finished — migrations are how you change a live database's shape safely, in version control, without losing data. intermediate 9 min

Unit 4 — Beyond Relational

The other database families and when they beat a table: key-value and document stores, time-series and analytics, vector search, and caching layers.

  1. SQL vs. NoSQL What "NoSQL" actually means, the tradeoff it makes, and how to tell when a non-relational store fits a problem better than tables do. intermediate 9 min
  2. Key-value & document stores Two of the most common NoSQL shapes — the dictionary-like key-value store and the JSON-document store — and the jobs each is built for. intermediate 9 min
  3. Time-series & analytical stores Databases tuned for when-it-happened data and for crunching huge tables — time-series and column stores, and why your metrics live in one. intermediate 9 min
  4. Vector databases & similarity search The store behind AI retrieval — embeddings as vectors, searching by meaning instead of keywords, and how RAG features lean on it. intermediate 10 min
  5. Caching layers Keeping hot data in fast memory in front of the database — caches like Redis, what they buy you, and the classic hard problem of keeping them fresh. intermediate 9 min

Unit 5 — Using a Database From Code

Wiring a database into a program: connecting, pooling connections, ORMs versus raw SQL, defending against injection, and doing it all in Go.

  1. Connecting from your program Drivers, connection strings, and credentials — how a running program actually opens a link to a database and sends it queries. intermediate 9 min
  2. Connection pools Opening a connection is expensive, so real apps reuse a pool of them — what a pool is, why you need one, and how to size it. intermediate 8 min
  3. ORMs vs. raw SQL The perennial choice — let a library map objects to rows, or write SQL by hand — what each buys you and what each costs. intermediate 9 min
  4. SQL injection & querying safely The classic, still-common vulnerability where user input becomes SQL — how it happens and why parameterised queries end it. intermediate 9 min
  5. Talking to a database from Go The Go way — database/sql, drivers, prepared statements, and querying safely — with the patterns GopherTrunk-style services use. advanced 10 min

Unit 6 — Running a Database in Production

Everything applied: keeping data safe with backups, scaling with replication, watching performance, choosing the right database, and the data behind GopherTrunk.

  1. Backups & recovery The single most important operational habit — backups you have actually restored, point-in-time recovery, and testing them before you need them. intermediate 9 min
  2. Replication, sharding & scaling When one server isn't enough — read replicas, sharding, and the CAP tradeoffs that shape how a database scales out. advanced 10 min
  3. Performance tuning & monitoring Finding the slow query before your users do — EXPLAIN, query plans, the metrics that matter, and the usual suspects behind a sluggish database. advanced 10 min
  4. Choosing a database SQLite, Postgres, MySQL, a document store, a managed service like Supabase — a practical framework for picking the right one for a project. intermediate 9 min
  5. Data in GopherTrunk A worked example — how a scanner like GopherTrunk models systems, talkgroups, and decoded calls, and stores them for search and playback. advanced 10 min
  1. Glossary of database terms Plain-language definitions for every term in the module — database, schema, SQL, primary and foreign key, join, index, normalization, transaction, ACID, NoSQL, vector store, ORM, migration, replica, and more — cross-linked to the lessons.