The planner
plan_diff(diff, dialect) turns the diff into an ordered list of Steps. A step has a description, one or more statements, a set of hazards and a transactional flag (False means "must run outside a transaction block").
Rename handling comes first
Before diffing structure the diff engine resolves renames and then applies them to a copy of the source schema (rename_schema): table names, FK targets, constraint columns, index columns and expressions, CHECKs and generated expressions are rewritten. The structural diff of that renamed copy against the target therefore sees no change for things that merely followed a rename, and the plan starts with ALTER TABLE ... RENAME steps.
Candidates:
- Column: removed column and added column in the same table with the same type, not-null, default, generation and the same ordinal position. Only unambiguous one-to-one matches are suggested.
- Table: removed table and added table whose column-name sets have Jaccard similarity >= 0.8 (
|A ∩ B| / |A ∪ B|), matched greedily by best score.a(x,y,z,w,v)tob(x,y,z,w,v,u)scores 5/6 = 0.83 and is suggested;a(x,y,z,w,v)tob(x,y,z,p,q)scores 3/7 and is not.
A candidate is applied only with --accept-rename old=new / table.old=new or after an interactive yes. Accepted renames that are already done (re-running apply) are ignored.
Ordering
Phases, in order:
- create schemas, enums, sequences; add enum values
- drop triggers, then views (dependents first)
- drop foreign keys, then other constraints; drop indexes
- renames (tables, then columns)
- create new tables in FK dependency order
- alter existing tables: columns first for all tables, then constraints
- create indexes
- drop removed tables in reverse dependency order
- drop removed sequences and enums
- create views (dependencies first), triggers; comments
Why: a column of enum type needs the type to exist first; a view must not exist while the column it reads changes type or disappears; an FK must go before the table or unique key it points at; a new FK may point at a column added earlier in the same plan.
Cycles: new tables that reference each other are created without the FKs that close the cycle (table_order picks the victim with the fewest unmet dependencies); those FKs are added afterwards with ALTER TABLE ... ADD CONSTRAINT.
Views referencing a table are found by scanning FROM/JOIN lists (CTE names excluded). Views are dropped and recreated when they changed, were removed, or (transitively) read a table whose column is altered/dropped (PostgreSQL) or a table that is rebuilt (SQLite).
PostgreSQL patterns
| Change on an existing table | Emitted |
|---|---|
| new index | CREATE INDEX CONCURRENTLY (outside a transaction); plain CREATE INDEX on a table created in the same plan |
| drop index | DROP INDEX CONCURRENTLY IF EXISTS |
| new FK | ADD CONSTRAINT ... NOT VALID, then VALIDATE CONSTRAINT as separate steps |
| new CHECK | same NOT VALID / VALIDATE pair |
| new UNIQUE / PRIMARY KEY | CREATE UNIQUE INDEX CONCURRENTLY, then ADD CONSTRAINT ... USING INDEX |
SET NOT NULL |
ADD CONSTRAINT c CHECK (col IS NOT NULL) NOT VALID, VALIDATE, SET NOT NULL (uses the validated check, no scan), drop the helper check; four steps, none holding a long lock |
| type change | ALTER COLUMN TYPE ... USING col::type (through text for enums), hazards by type_change_hazards |
| enum value added | ALTER TYPE ... ADD VALUE IF NOT EXISTS ... BEFORE/AFTER (outside a transaction) |
| generated expression changed | drop and re-add the column |
apply uses an autocommit connection with SET lock_timeout; transactional steps run inside conn.transaction(). The plan file groups consecutive transactional steps in BEGIN; ... COMMIT;.
Worked example
Making users.full_name NOT NULL on a big table:
alter table users add constraint users_full_name_not_null_tmp check (full_name is not null) not valid; -- brief lock
alter table users validate constraint users_full_name_not_null_tmp; -- scans, but only blocks DDL
alter table users alter column full_name set not null; -- trusts the valid check
alter table users drop constraint users_full_name_not_null_tmp;
Down migrations
build_plan plans the reverse diff (target -> source) with the renames inverted. Where PostgreSQL cannot go back (removing enum values) the value is left in place and a warning is attached; otherwise the down plan is complete. The -- migrate:down section lists the hazards of the reverse plan as comments (they do not gate plan; for example dropping a column that the up migration added is flagged DATA_LOSS).
Plan identity
Plan.hash is the SHA-256 of all statements; apply records it with the step count in _schemashift_history.