Glossary
Each track has its own glossary, because the same word can mean different things in different fields. General terms used everywhere are listed below.
General terms · shared by every track
- ACID
- Four guarantees for transactions: Atomic (all or nothing), Consistent (rules hold), Isolated (concurrent work doesn't interfere), Durable (once done, it stays done).
- Learn it in “ACID on object storage” →
- Atomic
- All or nothing. An atomic change either happens completely or not at all, and nobody ever sees it half done.
- Think of it as: Handing over a sealed envelope: the other person has it or doesn't. There's no half-handed envelope.
- Learn it in “Object storage: the ground floor” →
- Batch processing
- Processing data in scheduled chunks, such as every night or every hour.
- Learn it in “Getting data in” →
- CSV
- Comma-separated values: plain text, one row per line. Easy to read, slow and large for analytics, and no schema.
- Learn it in “Rows vs columns: file formats” →
- Idempotent
- Safe to run twice: running the job again gives the same result instead of duplicating data.
- Learn it in “Medallion architecture” →
- JSON
- A text format of nested key-value objects, like {"id": 7}. Common for app events and APIs, and used by Delta's transaction log.
- Learn it in “Rows vs columns: file formats” →
- Metadata
- Data about data: which files exist, their schema, sizes and statistics. Engines read metadata first to plan work.
- Learn it in “What makes a table a table” →
- Schema
- The structure of a table: its column names and types, such as order_id: long, amount: decimal.
- Learn it in “Schema evolution & enforcement” →
- SQL
- The standard language for querying tables: SELECT … FROM … WHERE …
- Learn it in “Hands-on: query in your browser” →
- Streaming
- Processing data continuously as it arrives, within seconds or minutes.
- Learn it in “Getting data in” →
- Transaction
- A group of changes that succeeds or fails as one unit.
- Think of it as: A money transfer: debit one account and credit the other, or do neither.
- Learn it in “ACID on object storage” →
A
B
C
I
J
M
S
T