Make your life easier by running a setup function in all your notebooks.

A few decoratos and general functions to add important information to the most fragile operations.

Developing DAGs locally.

I switched my blog to pelican.

Analytics SQL queries tend to get rather complex. To keep me sane, it is important to improve the development experience with tools like sqlfluff that can format and lint the queries. The following article describes the usage of sqlfluff with Jinja 2 templates (obligatory: don't use untrusted input for you …

Alright, there are 100 ways in Python land to do dependency management and it seems the mind share of projects shifts every two years. So, whatever, a summary of how I like to use poetry.

Installation

Poetry should not be installed in the same virtual environment as the project, because …

Python logging 101.

Dipping our feet into using IRC today.

Infrastructure as code allows for fully automatic infrastructure provisioning, often by declaratively defining a desired state of your infrastructure in some language. One tool to achieve this is terraform. The entry barrier is very low - a basic project setup is done in no time. But beyond the basics, some additional …

Alright, I had heard of fzf a long time ago, but I never took the time to really understand why I would want to use it. Well, I have been missing out.

Installation and configuration

On Arch, install fzf. Then, to make it useful from the terminal, activate the key …

Being in-between jobs at the moment, I decided to set up a private AWS account. Unfortunately, unlike Azure, Amazon does not offer an account with a limited budget. This means that in the worst case you can literally get ruined for life if your credentials are leaked, or if you …

With GNU parallel we can quickly parallelize any shell command. The documentation is extensive, so the following are my getting started notes.

In general, the syntax is:

parallel [parallel-options] [command] [command-arguments] [::: input]

There are a hundred books about how to hold an effective retrospective, which I am sure are worthwhile reading. But in my role I usually do not host a project retrospective myself, so the following are my short notes as a kick start for the future:

Roughly, a retrospective consists …

I don't do networking every day, so sometimes I need to look up the very basics again. So this is for my future self.

IP ranges are often defined in CIDR notation. E.g., using IPv4: 129.69.69.15/18. What does this mean? An IPv4 ip consist of …

When talking about cloud storage, the term 'object storage' pops up frequently. In a nutshell, the objects an object storage manages consist of the actual data, a key and some metadata. See wikipedia for some more information. Some examples for cloud object storage systems are AWS S3 (simple storage service …

Matlab can identify the current worker id with get(getCurrentTask(), 'ID'). While Octave does not even natively support parallel computing, the parallel package offers some parallel capabilities, but does not support labindex. So, I hacked my own solution (that currently only works on Linux).

tl;dr: give me the code …

My whole website is tracked in a git repository that I host privately on sourcehut. A clone of this repository exists on my web host from which this site is served. To synchronize both repositorries I use webhooks.

A webhook is basically a callback URL. My web host offers a …

I finally decided to move to a new web host (netcup.de). Hetzner's whole webhosting setup with the konsoleh management interface just feels super outdated. Netcup uses plesk and compared to konsoleh at least it is a huge step up. Moving my domain was painless and almost instant after I …

Nowadays (since 2018?) it is possible to run OpenSSH natively on Windows (see here for installation instructions). Moreover, OpenSSH can be configured via a configuration file (~/.ssh/config) which allows us to use aliases and set specific settings per host. But how can we use the same SSH configuration for …

Recently, I created an Azure deployment pipeline for a Python flask application. But the actual deployment step took 10 minutes, which I found unexpectedly long.

How to speed up the deployment step

We are talking about Azure pipelines here (the yaml flavour). There are different deployment methods that you can …

My .de domain is hosted on Hetzner, a German hosting company. I have the cheapest web hosting package they offer, called 'level1'. It doesn't give you full ssh access, so how can you deploy your (in my case static) page automatically? I came up with the following instructions that worked …

I have been wanting to have my own little website for many years now. So, finally: welcome! For now, this mainly serves as a digital business card.

In the future, my plan is to use this site as a notebook for my technical struggles, findings and revelations. Maybe some of …