Automating rollback of database migrations in your Django deployment pipeline

tl;dr: Scroll to the end of the page, and you will find two one-liners that help you record and roll back Django migrations.

Even though Django has nice built-in support for creating and maintaining database migrations, I find it surprising that one feature seems to be missing: There is no easy way to automatically roll back the migrations during a failed deployment.

I have the following workflow in mind for a typical automated, failed deployment.:

  1. Start automatic deployment
  2. Record which migrations will be applied, for possible rollback
  3. Apply migrations
  4. Deploy new application version
  5. Run smoke-tests
  6. Because the smoke-tests fail, rollback migrations that where recorded in step 2
  7. Re-deploy previous application version
  8. End automatic deployment of new version

The missing Django-pieces are step 2 und step 6. We are almost there with python manage.py showmigrations --plan, which will print a list of migrations that will be applied. The output looks like this:

[ ]  contenttypes.0001_initial
[ ]  contenttypes.0002_remove_content_type_name
[ ]  auth.0001_initial
[ ]  auth.0002_alter_permission_name_max_length
[ ]  auth.0003_alter_user_email_max_length
[ ]  auth.0004_alter_user_username_opts
[ ]  auth.0005_alter_user_last_login_null

All that is missing is a way to pipe this output back into python manage.py migrate <APP_NAME> <TARGET_MIGRATION> in reverse order, i.e.

$ python manage.py migrate auth 0005_alter_user_last_login_null
$ python manage.py migrate auth 0004_alter_user_username_opts
$ python manage.py migrate auth 0003_alter_user_email_max_length

etc.

Finding a Unix shell solution AKA “Let me google that for you”

I am terrible with writing bash-scripts. I have to Google everything if I ever need to write one. The only thing I always remember is to add set -eu at the start of each script to make sure that errors and missing environment variables don’t trip me up (set -eu will quit immediately once the script encounters an error or a missing environment variable).

But this shouldn’t be so bad, right?

What we need to do:

  • store the output in a file, let’s call it planned_migrations.txt
  • reverse the lines of the file
  • remove the first five characters [ ]
  • replace the dot with a space, i.e. auth.0005_alter_user_last_login_null becomes auth 0005_alter_user_last_login_null
  • call python manage.py migrate and pass the two strings from the previous step as parameters

Store the output in planned_migrations.txt

That is simple, even I can do that without Google:

$ python manage.py showmigrations --plan > planned_migrations.txt

Reverse the line of the file

If you have core-utils installed you can use tac (which is the reverse of cat). An alternative that also works on OSX is

$ tail -r planned_migrations.txt

Remove the first five characters

A quick Google showed that this can be achieved using cut -c6-. Note that cut seems to have some problems with non-ascii characters, so bare that in mind. The obvious but more complex alternative would be sed (sed 's/^.....//' seems to work). Let’s be on the safe side and use sed:

$ tail -r planned_migrations.txt | sed 's/^.....//'

Remove the period in each line

This is what tr (“translate”) can be used for. Note that this will fail if your migration-names have more than one period. In that case you will need to google yourself. :)

$ tail -r planned_migrations.txt | sed 's/^.....//' | tr . " "

**Run migration for each line in the file

The tool of choice seems to be ``xargs`.

$ tail -r planned_migrations.txt | sed 's/^.....//' | tr . " " | xargs -n 2 python manage.py migrate

The final result

Here is the result of my googling session. Please note that I have not done much testing, so be careful and always backup your production database!

Store the planned migrations in a text-file

$ python manage.py showmigrations --plan > planned_migrations.txt

Roll back all migrations that have been recorded in planned_migrations.txt

$ tail -r planned_migrations.txt | sed 's/^.....//' | tr . " " | xargs -n 2 python manage.py migrate
python  django 

If you liked this post, please, do share it:

Thanks, for reading (and sharing)! 🥳

See also