Debugging postgres: List all connections

Sometimes want a list of all connections to your postgres database. Postgres maintains this list in an internal datastructure that you simply query like this: SELECT * FROM pg_stat_activity; This will give you something like this (I have removed many columns to make it more readable here): state application_name client_addr wait_event NULL NULL AutoVacuumMain NULL NULL LogicalLauncherMain idle test. [Read More]

RxPY3: Using multiple observers with the buffer-operator

I created a github-repository with the two rxpy (version 3) examples from this and another previous blog-post. You can find it at rxpy3-examples on github. My previous example of the rxpy3 buffer-operator demonstrated the basic usage of the buffer-operator by using two interval-observables. In today’s example I will create a custom observable. While this particular observable is very simple, it demonstrates how to create an observable from a non-reactive data-source. [Read More]

RxPY: How to use the buffer-operator

I created a github-repository with the two rxpy (version 3) examples from this and another previous blog-post. You can find it at rxpy3-examples on github. When I was trying to understand and use the buffer operator of RxPy3, I noticed that there are not a lot of code samples out there. The few examples are usually for version 2 of RxPy, and that does not help if you are a complete RxPy3 n00b, like me. [Read More]

RxPy - Reactive Python

I want to publish my daily blog-post, like every day in the last weeks, but I don’t have the time to write a concrete code example. I have been tinkering with RxPy 3, which is the Python version of the Rx-library, an implementation of the Observer/Subscriber pattern, also sometimes called reactive programming. Using RxPy (or any of the Rx-flavors), forces you to view everything as a stream of events and/or data. [Read More]

Using PostgreSQL as a task queue

Many applications need some form of reliable background processing, usually in form a persistent queue. Instead of using a full-blown task queue like the popular celery-framework for Python, I like to use something that most projects already use anyway: PosgreSQL. For django-projects I like to use django-postgres-queue. For other projects I am experimenting with pq. PostgreSQL’s provides two facilities that make implementing a task queue feasible in the database: SKIP LOCKED and NOTIFY/LISTEN. [Read More]

Python progress bars and colors in a terminal

Text-based progress bars If you ever need to create a progress bar in a Python script or terminal application there is nifty little library called progress. I especially like the wrapper around iterables: 1 2 3 4 5 from import Bar for item in Bar('Processing').iter(items): # Process the item your_processing_function(item) which will show a nice, text-based progress bar: Processing |############# | 42/100 Terminal colors And if you want to add colors to your Python output, colorama is the way to go. [Read More]

Using *.env-files everywhere

Many tools automatically look for a special file called .env, which contains KEY=VALUE pairs, one per line. I am here referring to a file with the same syntax, but with arbitrary names. I use different names to distinguish between configurations for different development stages. Using environment variables for configuration is pretty much accepted as good practice today for web applications. But it’s always a hassle to deal with them during development, if you are not using a specialised service for them. [Read More]

Automating rollback of database migrations in your Django deployment pipeline

tl;dr: Scroll to the end of the page, and you will find two one-liners that help you record and roll back Django migrations. Even though Django has nice built-in support for creating and maintaining database migrations, I find it surprising that one feature seems to be missing: There is no easy way to automatically roll back the migrations during a failed deployment. I have the following workflow in mind for a typical automated, failed deployment. [Read More]

Using automatic security scans to improve your python code quality

I am a big fan of code scanning tools that help you improve the quality of your code-base. I have already written about import-linter, which helps to adhere to your architecture by validating imports. Another great tool is bandit. It scans your python code for common security problems. When I ran it on MembershipNerd, I got the following warning: Test results: >> Issue: [B106:hardcoded_password_funcarg] Possible hardcoded password: 'secret' Severity: Low Confidence: Medium Location: . [Read More]