Using PostgreSQL as a task queue

Many applications need some form of reliable background processing, usually in form a persistent queue. Instead of using a full-blown task queue like the popular celery-framework for Python, I like to use something that most projects already use anyway: PosgreSQL.

PostgreSQL’s provides two facilities that make implementing a task queue feasible in the database: SKIP LOCKED and NOTIFY/LISTEN. Both libraries use these features in their implementation.

I have not performed any benchmarks, because my typical workload is rather low, like sending password-reset emails. However, pq says: “In terms of performance, the implementation clock in at about 1,000 operations per second.” That’s plenty for many use cases.

See also