Debugging postgres: List all connections

Sometimes want a list of all connections to your postgres database. Postgres maintains this list in an internal datastructure that you simply query like this: SELECT * FROM pg_stat_activity; This will give you something like this (I have removed many columns to make it more readable here): state application_name client_addr wait_event NULL NULL AutoVacuumMain NULL NULL LogicalLauncherMain idle test.trader.PersistentQueue(data) 172.20.0.5 ClientRead active PyCharm 2021.2.3 172.20.0.1 NULL NULL NULL BgWriterHibernate NULL NULL CheckpointerMain NULL NULL WalWriterMain In this example, there are two client connections: one from PyCharm, one from my test-application showing up as “test. [Read More]

RxPY3: Using multiple observers with the buffer-operator

I created a github-repository with the two rxpy (version 3) examples from this and another previous blog-post. You can find it at rxpy3-examples on github. My previous example of the rxpy3 buffer-operator demonstrated the basic usage of the buffer-operator by using two interval-observables. In today’s example I will create a custom observable. While this particular observable is very simple, it demonstrates how to create an observable from a non-reactive data-source. [Read More]

RxPY: How to use the buffer-operator

I created a github-repository with the two rxpy (version 3) examples from this and another previous blog-post. You can find it at rxpy3-examples on github. When I was trying to understand and use the buffer operator of RxPy3, I noticed that there are not a lot of code samples out there. The few examples are usually for version 2 of RxPy, and that does not help if you are a complete RxPy3 n00b, like me. [Read More]

RxPy - Reactive Python

I want to publish my daily blog-post, like every day in the last weeks, but I don’t have the time to write a concrete code example. I have been tinkering with RxPy 3, which is the Python version of the Rx-library, an implementation of the Observer/Subscriber pattern, also sometimes called reactive programming. Using RxPy (or any of the Rx-flavors), forces you to view everything as a stream of events and/or data. [Read More]

Using PostgreSQL as a task queue

Many applications need some form of reliable background processing, usually in form a persistent queue. Instead of using a full-blown task queue like the popular celery-framework for Python, I like to use something that most projects already use anyway: PosgreSQL. For django-projects I like to use django-postgres-queue. For other projects I am experimenting with pq. PostgreSQL’s provides two facilities that make implementing a task queue feasible in the database: SKIP LOCKED and NOTIFY/LISTEN. [Read More]

Python progress bars and colors in a terminal

Text-based progress bars If you ever need to create a progress bar in a Python script or terminal application there is nifty little library called progress. I especially like the wrapper around iterables: 1 2 3 4 5 from progress.bar import Bar for item in Bar('Processing').iter(items): # Process the item your_processing_function(item) which will show a nice, text-based progress bar: Processing |############# | 42/100 Terminal colors And if you want to add colors to your Python output, colorama is the way to go. [Read More]

Using *.env-files everywhere

Many tools automatically look for a special file called .env, which contains KEY=VALUE pairs, one per line. I am here referring to a file with the same syntax, but with arbitrary names. I use different names to distinguish between configurations for different development stages. Using environment variables for configuration is pretty much accepted as good practice today for web applications. But it’s always a hassle to deal with them during development, if you are not using a specialised service for them. [Read More]

Automating rollback of database migrations in your Django deployment pipeline

tl;dr: Scroll to the end of the page, and you will find two one-liners that help you record and roll back Django migrations. Even though Django has nice built-in support for creating and maintaining database migrations, I find it surprising that one feature seems to be missing: There is no easy way to automatically roll back the migrations during a failed deployment. I have the following workflow in mind for a typical automated, failed deployment. [Read More]

Using automatic security scans to improve your python code quality

I am a big fan of code scanning tools that help you improve the quality of your code-base. I have already written about import-linter, which helps to adhere to your architecture by validating imports. Another great tool is bandit. It scans your python code for common security problems. When I ran it on MembershipNerd, I got the following warning: Test results: >> Issue: [B106:hardcoded_password_funcarg] Possible hardcoded password: 'secret' Severity: Low Confidence: Medium Location: . [Read More]