Python

Sometimes want a list of all connections to your postgres database. Postgres maintains this list in an internal datastructure that you simply query like this:

SELECT * FROM pg_stat_activity;

This will give you something like this (I have removed many columns to make it more readable here):

In this example, there are two client connections: one from PyCharm, one from my test-application showing up as “test.trader.PersistentQueue(data)”. The other rows are postgres internals that I don’t know about.

The 2021 Python developer is live. It takes about 15 minutes to complete.

Here are last year’s results.

I created a github-repository with the two rxpy (version 3) examples from this and another previous blog-post. You can find it at rxpy3-examples on github.

My previous example of the rxpy3 buffer-operator demonstrated the basic usage of the buffer-operator by using two interval-observables. In today’s example I will create a custom observable. While this particular observable is very simple, it demonstrates how to create an observable from a non-reactive data-source.

The relevant part you would need to adapt are these:

I created a github-repository with the two rxpy (version 3) examples from this and another previous blog-post. You can find it at rxpy3-examples on github.

When I was trying to understand and use the buffer operator of RxPy3, I noticed that there are not a lot of code samples out there. The few examples are usually for version 2 of RxPy, and that does not help if you are a complete RxPy3 n00b, like me.

I want to publish my daily blog-post, like every day in the last weeks, but I don’t have the time to write a concrete code example.

I have been tinkering with RxPy 3, which is the Python version of the Rx-library, an implementation of the Observer/Subscriber pattern, also sometimes called reactive programming. Using RxPy (or any of the Rx-flavors), forces you to view everything as a stream of events and/or data. While this can be hard to wrap your had around in the beginning, it also provides unique abilities. For example, if you need to process any kind of event stream or data stream, I would encourage you to have look at RxPy.

Many applications need some form of reliable background processing, usually in form a persistent queue. Instead of using a full-blown task queue like the popular celery-framework for Python, I like to use something that most projects already use anyway: PosgreSQL.

• For django-projects I like to use django-postgres-queue.
• For other projects I am experimenting with pq.

PostgreSQL’s provides two facilities that make implementing a task queue feasible in the database: SKIP LOCKED and NOTIFY/LISTEN. Both libraries use these features in their implementation.

Text-based progress bars

If you ever need to create a progress bar in a Python script or terminal application there is nifty little library called progress. I especially like the wrapper around iterables:

1
2
3
4
5

from progress.bar import Bar

for item in Bar('Processing').iter(items):
  # Process the item
  your_processing_function(item)

which will show a nice, text-based progress bar:

Many tools automatically look for a special file called .env, which contains KEY=VALUE pairs, one per line. I am here referring to a file with the same syntax, but with arbitrary names. I use different names to distinguish between configurations for different development stages.

Using environment variables for configuration is pretty much accepted as good practice today for web applications. But it’s always a hassle to deal with them during development, if you are not using a specialised service for them. My current approach is to create separate *.env-file for the development and testing stage: development.env and testing.env. These files use regular bash-syntax for variable declaration:

tl;dr: Scroll to the end of the page, and you will find two one-liners that help you record and roll back Django migrations.

Even though Django has nice built-in support for creating and maintaining database migrations, I find it surprising that one feature seems to be missing: There is no easy way to automatically roll back the migrations during a failed deployment.

I have the following workflow in mind for a typical automated, failed deployment.:

I am a big fan of code scanning tools that help you improve the quality of your code-base. I have already written about import-linter, which helps to adhere to your architecture by validating imports. Another great tool is bandit. It scans your python code for common security problems. When I ran it on MembershipNerd, I got the following warning:

Test results:
>> Issue: [B106:hardcoded_password_funcarg] Possible hardcoded password: 'secret'
   Severity: Low   Confidence: Medium
   Location: ./src/testhelpers/users.py:10
   More Info: https://bandit.readthedocs.io/en/latest/plugins/b106_hardcoded_password_funcarg.html
9	
10	SUPERUSER = UserCredentials(email="admin@example.com", password="secret")

The scanner identified a problem because hard-coded default value for a parameter called “password” could be a potential security issue. In this case, it is not an issue, because this is only part of a test-suite.