Tiny Database Patcher¶ ↑
tdp
is Tiny Database Patcher, and all it does it takes a set of database patches in a form of plain-text SQL files and applies them to the database.
Introduction¶ ↑
To use tdp
to apply schema definition to the database you need two things: you need the database and you need the schema definition.
Database¶ ↑
tdp
is built on top of Sequel and thus can use any database that Sequel supports (including Oracle, PostgreSQL, MySQL, SQL Server and others).
tdp
uses table named tdp_patch
for its internal book-keeping, take it into account when assigning permissions as well as if for some reason this name clashes with what you have in your own schema.
Schema definition¶ ↑
Schema definition is a set of plain-text files with SQL statements to manipulate the database.
tdp
differentiates between two kinds of patches: “volatile” and “permanent.” Permanent patches are ones that are intended to be applied only once, such as CREATE
/ ALTER
/ DROP TABLE
or updating production data, and reapplying them is deemed unsafe (i.e. may trigger errors or cause data corruption). Volatile patches are ones that are intended to be applied multiple times, such as CREATE OR REPLACE VIEW / FUNCTION
or inserting fixture data.
Putting it simple, if you make a new volatile patch, then apply it to the database, and then amend it, and tdp
detects discrepancy between what is in database and what is supposed to be in database, then it “assumes” that the right thing to do is to apply a new version, and that’s what it does during the upgrade. Whereas if you do the same with a permanent patch, tdp
“concludes” that it’s impossible to resolve the discrepancy, and so it triggers an error.
Convention for naming patch files is:
-
file with a permanent patch must have
.sql
extension and start with a number, e.g.001-initial-schema.sql
or201611001_add_accounts_table.sql
-
file with a volatile patch must have
.sql
extension and not start with a number (otherwise it’d be treated as a permanent patch), e.g.views.sql
orstored-procedures.sql
. -
files that don’t have
.sql
extension will be ignored.
The reason why files with permanent patches should have a number in their name is fairly simple: for this kind of changes the order in which they are applied is typically important (for instance, patch #1 creates a table, patch #2 adds a column to that table, patch #3 references that column and so on), and numbering provides an intuitive way to specify just that.
Upgrade procedure¶ ↑
Here is how tdp upgrade
is done:
-
Check that for each record in
tdp_patch
table there is a corresponding patch file. Trigger an error if anything is missing. -
Check that for each permanent patch in schema definition either SHA-1 in
tdp_patch
matches that of the file, or there’s no database record at all. -
Apply all new permanent patches (in alphanumeric order)
-
Apply all new and modified volatile patches (in alphanumeric order)
Installation¶ ↑
gem install tdp
Usage ¶ ↑
tdp
can be used as a library within Ruby code
require 'sequel' require 'tdp' DB = Sequel.connect('postgres://localhost/database_name', :user=>'user', :password=>'password' ) TDP.execute(DB) do |engine| engine << 'sql/schema' engine << 'sql/fixtures' engine.upgrade end
or as a command-line tool
$ tdp upgrade "postgres://user:password@host:port/database_name" sql/schema sql/fixtures
See Sequel documentation for details of how to connect to your database.
Commands¶ ↑
- bootstrap
-
Initializes database tables for keeping track of applied patches.
- upgrade
-
Applies all changes that need to be applied.
- retrofit
-
Erases existing data about applied patches and replaces it with information derived from provided schema definition. Useful when starting to use
tdp
with a legacy database or after code cleanups. - validate_upgradable
-
Runs same set of preliminary checks as
upgrade
. - validate_compatible
-
Validates that all patches are applied to the database and triggers an error if there’s a mismatch. Useful to do during startup of an application that uses the database to make sure that database indeed matches the application’s expectations.
- rename
-
Amends the data about applied patches after they were renamed (without content changes) in the configuration.
Warning¶ ↑
Using tdp
is about as safe and error-prone as applying patches manually. Make sure you did sufficient testing and have necessary backups before upgrading databases with sensitive data.