Free Software, Free Society!
Thoughts of the FSFE Community (English)

Monday, 20 January 2020

The importance of culture

Origin Post on LinkedIn, Published on January 6, 2020


Being abroad in Japan the last couple weeks, I’ve noticed that the high efficiency -from crossing roads to almost everything- they do (cooking/public transportation/etc) is due to the fact of using small queues for every step of the process. Reaching to a maximum throughout with small effort.

The culture of small batches/queues reminds me the core principles of #DevOps as they have identified in the book “The Goal: A Process of Ongoing Improvement” by Eli Goldratt and of course in “Theory of Constraints”.

Imagine this culture to everything you do in your life. From work to your personal life. Reducing any unnecessary extra cost, reducing waste by performing Kata. Kata is about form, from dancing to creating your cloud infrastructure with reproducible daily work or routines that are focusing in the process for reaching your business goals.

This truly impresses me in Japanese culture among with the respect they are showing to each other. You may of course notice the young people riding their bicycles in the middle of the street, watching their smartphone instead of the road đŸ˜€but the majority of people bow their head to show respect to other people and other people’s work or service.

We, sometimes forget this simple rule in our work. Sometimes the pressure, the deadlines or the plethora of open tickets in our Jira board (or boards) makes us cranky with our colleagues. We forget to show our respect to other people work. We forget that we need each other for reaching to our business values as a team.

We forget to have fun and joy. To be productive is not about closing tickets is about using your creativity to solve problems or provide a new or improve an old feature that can make your customers happy.

Is about the feedback you will get from your customers and colleagues, is about the respect to your work. Is about being happy.

For the first time in my life, I took almost 30days out of work, to relax, to detox (not having a laptop with me) to spend some time with family and friends. To be happy. So if any colleague from work is reading this article:

  • Domo arigato

Happy new year (2020) to everybody. I wish you all good health and happiness.

PS: I am writing this article in a superexpress speed train going to Hiroshima, at 300 km/h

Thursday, 16 January 2020

KPatience added to flathub. Which app should be next?

This week we added KPatience to flathub.

That makes for a quite a few applications from KDE already in flathub

Which one do you think we should add next?

Sunday, 12 January 2020

Magic wormhole – easiest way to transfer a file across the Internet

  • Seravo
  • 13:29, Sunday, 12 January 2020

Transferring files between two computers on the Internet is as old of a problem as the Internet itself, and surprisingly hard. Sending an attachment over e-mail involves all kind of hassles and does not work for big files. Having both the sending and receiving part sign up for Dropbox or a similar service, or setting up your own Nextcloud server requires unreasonably much work if you simply want to transfer just one file from one computer to another. Luckily we live now in 2020, and there is a solution: the Magic Wormhole, an open source software in Python by Brian Warner.

All you need to do is install it on both computers (e.g. apt install magic-wormhole) and run it. No user account or any other setup is required. It works across any networks, no need to have public IP addresses or anything.

To send a file, simply run wormhole send and the file name. To receive a file, just run wormhole receive, and enter the key phrase that given by the sending party.

Screenshot from sending party:

$ wormhole send 
Sending 3.7 MB file named ''
On the other computer, please run: wormhole receive
Wormhole code is: 7-virginia-drumbeat

Sending (->
100%|████████████████████| 3.75M/3.75M [00:01<00:00, 2.74MB/s]
File sent.. waiting for confirmation
Confirmation received. Transfer complete.

Screenshot from receiving party:

$ wormhole receive
Enter receive wormhole code: 7-virginia-drumbeat
 (note: you can use <Tab> to complete words)
Receiving file (3.7 MB) into:
ok? (y/N): y
Receiving (->
100%|████████████████████| 3.75M/3.75M [00:02<00:00, 1.81MB/s]
Received file written to

Simple and brilliant!

Sunday, 05 January 2020

Big Distro

I like to read GNU/Linux hobbyist forums from time to time. Partially to keep up with all the changes that are constantly happening within the lovely world of Free Software, but mostly because I'm just very excited about GNU/Linux. It is quite possibly the world's biggest international collaborative effort, and that's just mind-bogglingly cool—the idea that people from all over the world come together to make this amazing tool for everyone to freely use. And it works! Most of the time, anyway.

There is one thing that bothers me about the hobbyist forums, however, and that is:

btw i use arch

The prevalence of Arch Linux. Now I don't actually intensely dislike Arch Linux, and this post isn't “Ten Reasons Arch Linux Sucks”. It's a fine distribution that gets a lot of things right for the hobbyist crowd, and I am sure that it is a technologically sound distribution. This post isn't even about Arch Linux specifically—it is about the host of distributions with which Arch shares a lot of attention in the popularity contest. There is no immediate pattern that binds these distributions, but among them are Manjaro, Linux Mint, elementary OS, Solus, Zorin OS, Pop!_OS, NixOS, et cetera.

The crux is that I am a little sad that these distributions win out in the popularity contest. Generally speaking, these distributions serve very specific niches: A rolling release distribution model, a focus on a certain desktop environment, an experimental package manager, or some combination thereof. These distributions distinguish themselves very clearly, but it is my opinion that the best distribution distinguishes itself not in any single category, but in its general purposeness.

Or rather, that is a half-lie. General purposeness is a direct consequence of the main trait I seek in a distribution: Size. I am talking Big Distro. This is a plea for Debian, Fedora, openSUSE, and Ubuntu.

Size and general purposeness

When I talk about size, I'm not concerned about the amount of disk space the default disk image takes up. Rather, I'm honing in on a vague metric at the intersection of market share, project size, and the amount of packages. There is something that sets Debian, Fedora, and to a slightly lesser extent openSUSE and Ubuntu apart from all the other distributions—the sheer scope of these projects.

These projects are absolutely massive with hundreds of active contributors each. And the contributions aren't just limited to packaging; the projects have people working on internationalisation, infrastructure, support, new software development, quality assurance, outreach, documentation, design, accessibility, security, and the awe-inspiring task of coordinating all of this work.

As a result of collaboration at this massive scope, these distributions have an unmatched general purposeness. Just about anything you might want to do, you can do with these distributions, and you can be fairly certain that it's supported.

Contrast this with other distributions, and you'll find that they have much smaller teams supporting them. Arch Linux actually stands out here in having a sizeable contributor base, but Solus has only a handful of people actively working on it. Mind, this isn't necessarily indicative of quality, but certainly of scope.

But why does scope matter? Surely Solus is simply just good at what it does, which is providing a high-quality Budgie desktop, and doesn't need to do anything else.


The best example of scope being important is security. You simply need people working full-time on security if you're creating a distribution that you expect people to use for their privacy-sensitive computing. Certainly if I'm relying on an operating system, I get some peace of mind in knowing that there is a team of people that is actively trying to make sure that the whole thing is and stays secure.

Security is a daunting task, because security flaws can creep in anywhere. It isn't sufficient to simply use the latest version of all software and rely on upstream to get things right, because security flaws can be introduced by the way that distribution makers configure, combine, or distribute the software.

Although I don't intend to name-and-shame in this article, I think that the smaller distributions do a generally less-than-stellar job in the security department. Especially noteworthy is Linux Mint containing malware for a while because their website had been compromised. The linked LWN article is worth a read, and echoes some of the sentiments I am writing here:

The Linux Mint developers have taken a certain amount of grief for this episode, and for their approach to security in general. They do not bother with security advisories, so their users have no way to know if they are affected by any specific vulnerability or whether Linux Mint has made a fixed package available. Putting the web site back online without having fully secured it mirrors a less-than-thorough approach to security in general. These are charges that anybody considering using Linux Mint should think hard about. Putting somebody's software onto your system places the source in a position of great trust; one has to hope that they are able to live up to that trust.

It could be argued that we are approaching the end of the era of amateur distributions. Taking an existing distribution, replacing the artwork, adding some special new packages, and creating a web site is a fair amount of work. Making a truly cohesive product out of that distribution and keeping the whole thing secure is quite a bit more work. It's not that hard to believe that only the largest and best-funded projects will be able to sustain that effort over time, especially when faced with an increasingly hostile and criminal net.

Though, in the spirit of fairness, it goes on to add:

There is just one little problem with that view: it's not entirely clear that the larger, better-funded distributions are truly doing a better job with security. It probably is true that they are better able to defend their infrastructure against attacks, have hardware security modules to sign their packages, etc. But a distribution is a large collection of software, and few distributors can be said to be doing a good job of keeping all of that software secure.

Linux Mint is not the only distribution that has struggled with security. Manjaro let their SSL certificate expire not once, but twice, and suggested some questionable workarounds. Frustratingly, these two distributions are often recommended to beginners and laypeople.


Accessibility is important, and a lot of smaller distributions fail immensely on this front. Arch Linux is nearly impossible to use if you are technologically disinclined or have a disability that makes using a TTY terminal difficult. Strangely, some people see this as a strength of Arch Linux. I disagree firmly with this. At best, Arch Linux sacrifices accessibility to enhance or enable some of their niche goals. Its developers might justify this choice because non-technical and disabled people simply aren't their target audience.

But accessibility is important, and GNOME is the only desktop environment I can think of that takes accessibility absolutely seriously, followed by KDE Plasma. Incidentally, GNOME is the default desktop environment of three of the four Big Distros, and openSUSE ships both GNOME and KDE Plasma in their installation image.

Everything else is important, too

The other aspects of scope are a little difficult to individually highlight, but I think they are all important in a project. For example, both openSUSE and Fedora use openQA to test their distributions as a cohesive whole. This completely automated suite runs hundreds of tests, and catches bugs before humans do. At the risk of saying the obvious, quality assurance makes a distribution better, and bigger distributions have more resources to do good quality assurance.

And at the risk of repeating the obvious, X makes a distribution better, and bigger distributions have more resources to do X. Substitute X with internationalisation, infrastructure, support, outreach, documentation, design, accessibility, and so forth.

In conclusion to an earlier question: Solus is good at what it does, which is providing a high-quality Budgie desktop, but it would be a lot better if it had the resources to do everything else as well.

But it doesn't. And unless it grows to join the list of Big Distros, it won't.

But I'm not personally affected

An obvious retort would be that—barring perhaps security—none of that matters, because I'm happy with my favourite niche distribution! And there is little that can be said in response to that individually. If you're happy with a distribution, then keep doing what you're doing, and don't pay too much attention to an opinion-haver on the internet.

But I don't think that that retort is sufficient. You see, I want Free Software to actually succeed. I want to live in a world where Free Software has won. And towards that end, I don't think the smaller distributions are sufficient at all. A lot of work goes into creating a cohesive, all-encompassing distribution for the masses, and the likes of Linux Mint aren't up to that task.

It's the difference between “what would happen if I installed Linux Mint on my grandmother's computer?” and “what would happen if I installed Linux Mint on the computer of millions of laypeople?". Grandma is probably going to be just fine individually, but the masses are seriously underserved by an understaffed distribution.

I see GNU/Linux as the public technological infrastructure of the future. And towards that end, I think we can do better than fractured, tiny distributions that serve hyper-specific niches.

Footnote: Linux Mint is a derivative distribution

Because Linux Mint is 99% identical to Ubuntu owing to its derivative status, one might argue that it benefits both from the scope and size of the Ubuntu project as well as the additional expertise that goes into it. That would make a lot of the above arguments null and void, because you're basically using Ubuntu.

I want to argue instead that Linux Mint loses a lot of the benefits of the scope of Ubuntu. Linux Mint has to duplicate a lot of the effort that goes into Ubuntu. It obviously needs its own infrastructure, translations, design, and so forth. But it also needs its own quality assurance and security team. By introducing small changes to the cohesive whole, Linux Mint introduces a lot of vectors for errors and security flaws.

Moreover, Linux Mint changes the desktop environment, which is like the most important component for your average user. All of the quality assurance and accessibility work that Ubuntu and others put into GNOME does not apply to Linux Mint's Cinnamon. So on the contrary, you are not basically using Ubuntu. You are using Ubuntu with its most important component replaced. It's the difference between getting a car from a trusted car manufacturer, or that same car, but some hobbyists changed the entire interior.

Friday, 03 January 2020

Support for REUSE (SPDX) headers in emacs-reveal

About 18 months ago, I asked: Do you teach or educate?

I continue to use and develop emacs-reveal, a FLOSS bundle to create HTML presentations based on reveal.js as Open Educational Resources (OER) from Org mode source files in GNU Emacs. Last time, I mentioned license attribution for OER figures as tedious challenge, which I believe to be addressed properly in emacs-reveal.

Over the last couple of days, I added functionality that generates license information in my OER HTML presentations from SPDX headers embedded in source files. The FSFE project REUSE recommends the use of SPDX headers to indicate copyright and licensing information in free software projects, and, although OER are not software, I started to make my OER source files REUSE compliant. Thus, the following two simple header lines in an Org source file (e.g., the emacs-reveal howto)

#+SPDX-FileCopyrightText: 2017-2020 Jens Lechtenbörger <>
#+SPDX-License-Identifier: CC-BY-SA-4.0

result in the following HTML licensing information (as part of the final slide in the howto) with RDFa markup for machine readability:

<div class="rdfa-license" about="">
  <p>Except where otherwise noted, the work 
     “<span property="dcterms:title">How to create presentations with emacs-reveal</span>”,
     <span property="dc:rights">© 
        <span property="dcterms:dateCopyrighted">2017-2020</span>
        <a rel="cc:attributionURL dcterms:creator"
           property="cc:attributionName">Jens Lechtenbörger</a></span>, 
     is published under the 
     <a rel="license" href="">
       Creative Commons license CC BY-SA 4.0</a>.</p></div>

Previously, I created the slide with license information separately, without taking copyright and license information of source files into account. Clearly, that type of redundancy was a Bad Thing, which I got rid of now. And, since license information is generated (currently in English or German based on the document’s language), you as future users of emacs-reveal do not need to know anything about RDFa (and next to nothing about HTML).

Happy New Year to all of you!

Sunday, 29 December 2019

Re: The Ecosystem is Moving

Moxie Marlinspike, the creator of Signal gave a talk at 36C3 on Saturday titled “The ecosystem is moving”.

The Fahrplan description of that talk reads as follows:

Considerations for distributed and decentralized technologies from the perspective of a product that many would like to see decentralize.

Amongst an environment of enthusiasm for blockchain-based technologies, efforts to decentralize the internet, and tremendous investment in distributed systems, there has been relatively little product movement in this area from the mobile and consumer internet spaces.

This is an exploration of challenges for distributed technologies, as well as some considerations for what they do and don’t provide, from the perspective of someone working on user-focused mobile communication. This also includes a look at how Signal addresses some of the same problems that decentralized and distributed technologies hope to solve.

Basically the talk is a reiteration of some arguments from a blog post with the same title he posted back in 2016.

In his presentation, Marlinspike basically states that federated systems have the issue of being frozen in time while centralized systems are flexible and easy to change.

As an example, Marlinspike names HTTP/1.1, which was released in 1999 and on which we are stuck on ever since. While it is true that a huge part of the internet is currently running on HTTP 1.0 and 1.1, one has to consider that its successor HTTP/2.0 was only released in 2015. 4 / 5 years are not a long time to update the entirety of the internet, especially if you consider the fact that the big browser vendors announced to only make their browsers work with HTTP/2.0 sites when they are TLS encrypted.

Marlinspike then goes on listing 4 expectations that advocates of federated systems have, namely privacy, censorship resistance, availability and control. This is pretty accurate and matches my personal expectations pretty well. He then argues, that Signal as a centralized application can fulfill those expectations as well, if not better than a decentralized system.


Privacy is often expected to be provided by the means of data ownership, says Marlinspike. As an example he mentions email. He argues that even though he is self-hosting his emails, “each and every mail has GMail at the other end”.

I agree with this observation and think that this is a real problem. But the answer to this problem would logically be that we need to increase our efforts to change that by reducing the number of GMail accounts and increasing the number of self-hosted email servers, right? This is not really an argument for centralization, where each and every message is guaranteed to have the same service at the other end.

I also agree with his opinion that a more effective tool to gain privacy is good encryption. He obviously brings the point that email encryption is unusable, (hinting to PGP probably), totally ignoring modern approaches to email encryption like autocrypt.

Censorship resistance

Federated systems are censorship resistant. At least that is the expectation that advocates of federated systems have. Every time a server gets blocked, the user just simply switches to another server. The issue that Marlinspike points out is, that every time this happens, the user loses his entire social graph. While this is an issue, there are solutions to this problem, one being nomadic identities. If some server goes down the user simply migrates to another server, taking his contacts with him. Hubzilla does this for example. There are also import/export features present in most services nowadays thanks to the GDPR. XMPP offers such a solution using XEP-0277.

But lets take a look at how Signal circumvents censorship according to Marlinspike. He proudly presents Domain Fronting as the solution. With domain fronting, the client connects to some big service which is costly to block for a censor and uses that as a proxy to connect to the actual server. While this appears to be a very elegant solution, Marlinspike conceals the fact that Google and Amazon pretty quickly intervened and stopped Signal from using their domains.

With Google Cloud and AWS out of the picture, it seems that domain fronting as a censorship circumvention technique is now largely non-viable in the countries where Signal had enabled this feature.

Notice that above quote was posted by Marlinspike himself more than one and a half years ago. Why exactly he brings this as an argument remains a mystery to me.

Update: Apparently Signal still successfully uses Domain Fronting, just with content delivery networks other than Google and Amazon.

And even if domain fronting was an effective way to circumvent censorship, it could also be applied to federated servers as well, adding an additional layer of protection instead of solely relying on it.

But what if the censor is not a foreign nation, but instead the nation where your servers are located? What if the US decides to shutdown for some reason? No amount of domain fronting can protect you from police raiding your server center. Police confiscating each and every server of a federated system (or even a considerable fraction of it) on the other hand is unlikely.


This brings us nicely to the next point on the agenda, availability.

If you have a centralized service than you want to move that centralized service into two different data centers. And the way you did that was by splitting the data up between those data centers and you just halved your availability, because the mean time between failures goes up since you have two different data centers which means that it is more likely to have an outage in one of those data centers in any given moment.

Moxie Marlinspike in his 36c3 talk “The Ecosystem is Moving”

For some reason Marlinspike confuses a decentralized system with a centralized, but distributed system. It even reads “Centralized Service” on his slides… Decentralization does not equal distribution.

A federated system would obviously not be fault free, as servers naturally tend to go down, but an outage only causes a small fraction of the network to collapse, contrary to a total outage of centralized systems. There even are techniques to minimize the loss of functionality further, for example distributed chat rooms in the matrix protocol.


The advocates argument of control says that if a service provider behaves undesirably, you simply switch to another service provider. Marlinspike rightfully asks the question how it then can be that many people still use Yahoo as their mail provider. Indeed that is a good question. I guess the only answer I can come up with is that most people probably don’t care enough about their email to make the switch. To be honest, email is kind of boring anyways đŸ˜‰


Next Marlinspike talks about XMPP. He (rightfully) notes that due to XMPPs extensibility there is a morass of XEPs and that those don’t really feel consistent.

The XMPP community already recognized the problem that comes with having that many XEPs and tries to solve this issue by introducing so called compliance suites. These are annually published documents that contain a list of XEPs that are considered vitally important for clients or servers. These suites act as maps that point a way through the XEP jungle.

Next Marlinspike states that the XMPP protocol still fails to be a suitable option for mobile devices. This statement is plain wrong and was already debunked in a blog post by Daniel Gultsch back in 2016. Gultsch develops an XMPP client for Android which is totally usable and generally has lower battery consumption than Signal has. Conversations implements all of the XEPs listed in the compliance suites to be required for mobile clients. This shows that implementing a decent mobile client for a federated system can be done and there is a recipe for it.

What Marlinspike could have pointed out instead is that the XMPP community struggles to come up with a decent iOS client. That would have been a fair argument, but spreading FUD about the XMPP protocol as a whole is unfair and dishonest.

Luckily the audience of the talk didn’t fully buy into Marlinspikes weaker arguments as demonstrated by some entertaining questions during the QA afterwards.

What Marlinspike is right about though is that developing a federated system is harder than doing a centralized service. You as the developer have control over the whole system and subsequently over the users. However this is actually the reason why we, the community of decentralized systems and federated protocols do what we do. In the words of J.F. Kennedy, we do these things…

…not because they are easy, but because they are hard…

… or simply because they are right.

Friday, 27 December 2019

How to create an AppImage

AppImage is a brilliant way to have executable linux apps to every distro, without the need of re-packaging or re-build them. Without getting into too many details, it uses FUSE (Filesystem in Userspace) and SquashFS to bundle the app into one file.

AppImages require FUSE to run. Filesystem in Userspace (FUSE) is a system that lets non-root users mount filesystems.

So here are my personal notes on how to create Mozilla Firefox 68.3.0esr binary archive to an AppImage file.


Let’s begin by gathering all necessaries files

export VERSION=68.3.0esr

curl -sLO

curl -sL$VERSION/linux-x86_64/en-US/firefox-$VERSION.tar.bz2 | tar xjf -

configuration files

we need 3 files, under the firefox directory

  • AppRun (executable shell script)
  • Icon (.png,.svg,.xpm)
  • firefox.desktop ( desktop file)


this is our guide, this file will start our application inside the AppImage mount.

cd "$(dirname "$0")"
exec ./firefox "$@"


cat > firefox/AppRun <<EOF
cd "\$(dirname "\$0")"
exec ./firefox "\$@"


Dont forget to make it executable

chmod +x firefox/AppRun


There is an image within firefox directory that we can use as firefox icon:



for more info check here: Desktop Entry Specification

[Desktop Entry]
Name=Mozilla Firefox


cat > firefox/firefox.desktop <<EOF
[Desktop Entry]
Name=Mozilla Firefox

In the Icon attribute, it must be an absolute path, not relative.


Give execute permission to appimagetool

chmod +x appimagetool-x86_64.AppImage

Build your AppImage

./appimagetool-x86_64.AppImage --no-appstream firefox/

Mozilla Firefox

if everything is okay, you will see this:

ls -l Mozilla_Firefox-x86_64.AppImage

and you can run it !



if you want to run a specific profile:

./Mozilla_Firefox-x86_64.AppImage --profile $(pwd)/.mozilla/firefox/ichznbon.test/


When you are running your AppImage, you will notice that there is a new mount point in our system (fusermount)

$ mount | grep -i firefox
Mozilla_Firefox-x86_64.AppImage on /tmp/.mount_MozillshcmPB type fuse.Mozilla_Firefox-x86_64.AppImage (ro,nosuid,nodev,relatime,user_id=347,group_id=347)

and if you look really careful, you will see that it is mounted under /tmp/ !

$ ls /tmp/.mount_MozillshcmPB
application.ini     firefox          icons            minidump-analyzer     Throbber-small.gif
AppRun              firefox-bin      libfreeblpriv3.chk      omni.ja               updater
browser             firefox-bin.sig     pingsender            updater.ini
chrome.manifest     firefox.desktop        libsoftokn3.chk  platform.ini          update-settings.ini
crashreporter       firefox.sig   plugin-container
crashreporter.ini   fonts       libnssdbm3.chk       plugin-container.sig
defaults            gmp-clearkey        precomplete
dependentlibs.list  gtk2       removed-files

That’s it !

Your first AppImage bundle linux package.

Docker Notes

FUSE ¡ AppImage/AppImageKit Wiki ¡ GitHub

docker run --cap-add SYS_ADMIN --cap-add MKNOD --device /dev/fuse:mrw --rm -ti ubuntu:18.04 bash

 apt-get update

 apt-get -y install curl libfuse2 file 

 export VERSION=68.3.0esr

 curl -sLO

 curl -sL$VERSION/linux-x86_64/en-US/firefox-$VERSION.tar.bz2 | tar xjf -

 cat > firefox/AppRun <<EOF
cd "\$(dirname "\$0")"
exec ./firefox "\$@"

 cat > firefox/firefox.desktop <<EOF
[Desktop Entry]
Name=Mozilla Firefox

 chmod +x appimagetool-x86_64.AppImage

 ./appimagetool-x86_64.AppImage --no-appstream firefox/
appimagetool, continuous build (commit 64321b7), build 2111 built on 2019-11-23 22:20:53 UTC
WARNING: gpg2 or gpg command is missing, please install it if you want to create digital signatures
Using architecture x86_64
/firefox should be packaged as Mozilla_Firefox-x86_64.AppImage
Deleting pre-existing .DirIcon
Creating .DirIcon symlink based on information from desktop file
Generating squashfs...
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on Mozilla_Firefox-x86_64.AppImage, block size 131072.
[===========================================================================================================================|] 1583/1583 100%

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
    compressed data, compressed metadata, compressed fragments,
    compressed xattrs, compressed ids
    duplicates are removed
Filesystem size 71064.05 Kbytes (69.40 Mbytes)
    36.14% of uncompressed filesystem size (196646.16 Kbytes)
Inode table size 5305 bytes (5.18 Kbytes)
    60.46% of uncompressed inode table size (8774 bytes)
Directory table size 1026 bytes (1.00 Kbytes)
    54.78% of uncompressed directory table size (1873 bytes)
Number of duplicate files found 3
Number of inodes 81
Number of files 67
Number of fragments 7
Number of symbolic links  1
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 13
Number of ids (unique uids + gids) 1
Number of uids 1
    root (0)
Number of gids 1
    root (0)
Embedding ELF...
Marking the AppImage as executable...
Embedding MD5 digest

Please consider submitting your AppImage to AppImageHub, the crowd-sourced
central directory of available AppImages, by opening a pull request

final notes:

 du -h Mozilla_Firefox-x86_64.AppImage
70M Mozilla_Firefox-x86_64.AppImage

 ls -l Mozilla_Firefox-x86_64.AppImage
-rwxr-xr-x 1 root root 72962088 Dec 26 21:55 Mozilla_Firefox-x86_64.AppImage

 file Mozilla_Firefox-x86_64.AppImage
Mozilla_Firefox-x86_64.AppImage: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.18, stripped

 ldd Mozilla_Firefox-x86_64.AppImage
    not a dynamic executable
Tag(s): AppImage, firefox

Wednesday, 25 December 2019

doh-cli, a simple DoH client

original post on LibreOps

A couple months ago, we announced a public and free DNS service, so people can have encrypted DNS in their browsers and systems. We support both DNS over HTTPS (DoH) and DNS over TLS and our DoH service has two endpoints, the default /dns-query and one for blocking trackers and ads /ads. You can visit our page for more info.


What is DNS?

Domain Name Service in a nutshell is when you are asking directions to find where Wikipedia is in the internet. Your browser does not know, so it will ask your computer. Your computer will ask your internet provider and your internet provider will ask someone else till they find the correct answer. In the end, your browser will know where to go and this is how you are visiting Wikipedia.

You need to trust all the above parties, to give you the correct answer and everybody knows that you are visiting Wikipedia.


What is DoH (DNS Queries over HTTPS)?

It’s the implementation of RFC 8484. This is a way for your browser to ask where to find Wikipedia, without exposing to everybody that you want to visit Wikipedia! Still you need someone to ask for directions, but now both your question and the answer are encrypted. So you have privacy.

let’s get technical

What is RFC 8484?

In the above rfc, your client (eg. browser) asks your DNS via HTTP/2 representational state transfer (REST). DoH clients and servers need to sent a application/dns-message content (question/answer) and encode both the question and the answer in a bace64url message. Usually is GET, but POST is also supported on some servers.


So, today, we introduce doh-cli, a simple command line DoH client, written in python. You can use doh-cli as a binary client in your system. We support a few DoH public servers to test, and of course both LibreDNS DoH endpoints

You can see the code here:

install it

It is super easy

pip install doh-cli

or if python3 is not your default python

pip3 install doh-cli

howto use it?

Just ask your favorite DoH server (default is


doh-cli A

and use help to see all the options

doh-cli --help

Why default output is json?

With modern tools and with multiline output, it is best to support a serialized format so you can use doh-cli with your tools. But if you dont like it:

doh-cli --output plain A

You can see all the options and help, on the project’s page.


Tag(s): doh-cli, DoH, python

Tuesday, 24 December 2019

ipname - hostnames for all

A few day ago, I was introduced to
TLDR; You can have hostname for any IP Address!

$ dig +short



It uses the powerdns pipe backend to run a (187 lines) bash script, that strips the IP from the hostname and returns the IP. This works so well, that a few services depends on xip!

I was playing with the idea of using dnsdist to do that with the embedded Lua supports that dnsdist has. And the proof-of-concept result is about 10lines of Lua code.

The project is here: ipname on github


But not only returns you an IP Address for any (dynamic) hostname you ask, but you can also use this free & public service as a what-is-my-ip project over DNS.

$ dig +short

PS The code also validates the IPv4 Addresses!

Tag(s): ipname, dnsdist

Monday, 23 December 2019

Don't miss the new software freedom podcast

In October we have started to publish a podcast from the FSFE. Meanwhile we have three episodes: with Cory Doctorow, with Lydia Pintscher, and with Harald Welte. The plan for 2020 is to publish one around every month.

Software freedom podcast logo

Some years ago I myself started to listen to podcasts. Not so much podcasts about technology but rather documentaries, features, and comments about politics. I mainly did that while travelling or if I was not tired but wanted to relax my eyes a bit.

Earlier this year Katharina Nocun encouraged us to start a podcast for the FSFE ourselves. After some considerations we decided to give it a try and cover topics about software freedom in a monthly podcast.

In October for the International Day Against digital restriction management (DRM) I was happy that Cory Doctorow, one of my favourite writers agreed to join us as a guest. We talked with him about the difference between books and an e-books with DRM, how authors and artist can make money without DRM, security implications of DRM, regulation of the so called "Internet of Things", and other questions related to this issue.

In November we talked with Lydia Pintscher, vice president of KDE, about the development of the KDE community, the different KDE projects, the issues they will be tackling over the next two years, how to maintain long term sustainability in such a large project, and how she balances her long time volunteer commitment with her day job.

The last episode for this year was dedicated to an area which was in the focus of heated discussions during 2019: how should the new mobile phone infrastructure for 5G look like. Several countries have taken steps to ban specific vendors and try to convince others to do likewise. For this topic we got Harald Welte – Free Software developer for Osmocom ("open source mobile communications) whom many might know for his past work for and the OpenMoko. Harald gave an overview of the use of Free Software in mobile phone communication, basics of this technology, or the Huawei ban.

You can subscribe to the podcast either through the OPUS feed or MP3 feed so you do not miss new episodes next year.

If you are new to podcasts I got the feedback from many that they enjoy listing to podcasts on their mobile with Antennapod which you can install through F-Droid.

Friday, 13 December 2019

a simple DoH/DoT using only dnsdist

In this blog post I will describe the easiest installation of a DoH/DoT VM for personal use, using dnsdist.

Next I will present a full installation example (from start) with dnsdist and PowerDNS.

Server Notes: Ubuntu 18.04
Client Notes: Archlinux

Every {{ }} is a variable you need to change.
Do NOT copy/paste without making the changes.


Login to VM

and became root

$ ssh {{ VM }}
$ sudo -i

from now on, we are running commands as root.


dnsdist DoH/DoT

If you just need your own DoH and DoT instance, then dnsdist will forward your cleartext queries to another public DNS server with the below configuration.

cat > /etc/dnsdist/dnsdist.conf <<EOF

-- resets the list to this array

addDOHLocal('', '/etc/dnsdist/fullchain.pem', '/etc/dnsdist/privkey.pem')
addTLSLocal('', '/etc/dnsdist/fullchain.pem', '/etc/dnsdist/privkey.pem')


You will need -of course- to have your certificates before hand.
That’s It !

a DoH/DoT using dnsdist and powerdns

For people that need a more in-depth article, here are my notes on how to setup from scratch an entire VM with powerdns recursor and dnsdist.

Let’s Begin:

Enable PowerDNS Repos

Add key

curl -sL | apt-key add -

Create PowerDNS source list

cat > /etc/apt/sources.list.d/powerdns.list <<EOF
deb [arch=amd64] bionic-dnsdist-14 main
deb [arch=amd64] bionic-rec-42 main

cat > /etc/apt/preferences.d/pdns <<EOF
Package: pdns-* dnsdist*
Pin: origin
Pin-Priority: 600

Update System and Install packages

apt-get update
apt-get -qy install dnsdist pdns-recursor certbot

You may see errors from powerdns, like

  failed: E: Sub-process /usr/bin/dpkg returned an error code (1)

ignore them for the time being.

PowerDNS Recursor

We are going to setup our recursor first and let’s make it a little interesting.

PowerDNS Configuration

cat > /etc/powerdns/recursor.conf <<EOF

chmod 0644 /etc/powerdns/recursor.conf
chown pdns:pdns /etc/powerdns/recursor.conf

Create a custom response

This will be handy for testing our dns from cli.

cat > /etc/powerdns/pdns.lua <<EOF
domainame = "test.{{ DOMAIN }}"
response  = "{{ VM_ipv4.address }}"

function nxdomain(dq)
    if dq.qname:equal(domainame) then
        dq.rcode=0 -- make it a normal answer
        dq:addAnswer(pdns.A, response)
        dq.variable = true -- disable packet cache
        return true
    return false

chmod 0644 /etc/powerdns/pdns.lua
chown pdns:pdns /etc/powerdns/pdns.lua


Let’s make it more interesting, block trackers and ads.

cat > /usr/local/bin/ <<EOF

# Get StevenBlack hosts
curl -sLo /tmp/hosts.txt

touch /etc/powerdns/hosts.txt

# Get diff
diff -q <(sort -V /etc/powerdns/hosts.txt | column -t) <(sort -V /tmp/hosts.txt | column -t)

# Get Lines
LINES=`grep -c ^ /tmp/hosts.txt`

# Check & restart if needed
if [ "${LINES}" -gt "200" -a "${DIFF_STATUS}" != "0" ]; then
    mv -f /tmp/hosts.txt /etc/powerdns/hosts.txt
    chmod 0644 /etc/powerdns/hosts.txt
    chown pdns:pdns /etc/powerdns/hosts.txt
    systemctl restart pdns-recursor

# vim: sts=2 sw=2 ts=2 et

chmod +x /usr/local/bin/

Be Careful with Copy/Paste. Check the $ dollar sign.

OpenNic Project

Is it possible to make it more interesting ?
Yes! by using OpenNIC Project, instead of the default root NS

cat > /usr/local/bin/ <<EOF

# Get root hints
dig . NS @ | egrep -v '^;|^$' > /tmp/root.hints

touch /etc/powerdns/root.hints

# Get diff
diff -q <(sort -V /etc/powerdns/root.hints | column -t) <(sort -V /tmp/root.hints | column -t)

# Get Lines
LINES=`grep -c ^ /tmp/root.hints`

# Check & restart if needed
if [ "${LINES}" -gt "20" -a "${DIFF_STATUS}" != "0" ]; then
    mv -f /tmp/root.hints /etc/powerdns/root.hints
    chmod 0644 /etc/powerdns/root.hints
    chown pdns:pdns /etc/powerdns/root.hints
    systemctl restart pdns-recursor

# vim: sts=2 sw=2 ts=2 et

chmod +x /usr/local/bin/


dnsdist is a DNS load balancer with enhanced features.

dnsdist configuration

cat > /etc/dnsdist/dnsdist.conf <<EOF
-- resets the list to this array

addDOHLocal('', '/etc/dnsdist/fullchain.pem', '/etc/dnsdist/privkey.pem')
addTLSLocal('', '/etc/dnsdist/fullchain.pem', '/etc/dnsdist/privkey.pem')



Now it is time to get a new certificate with the help of letsencrypt.

Replace {{ DOMAIN }} with your domain

We need to create the post hook first and this is why we need to copy the certificates under dnsdist folder.

cat > /usr/local/bin/ <<EOF

cp -f /etc/letsencrypt/live/{{ DOMAIN }}/*pem /etc/dnsdist/
systemctl restart dnsdist.service

# vim: sts=2 sw=2 ts=2 et

chmod +x /usr/local/bin/

and of course create a certbot script.

Caveat: I have the dry-run option in the below script. When you are ready, remove it.

cat > /usr/local/bin/ <<EOF

certbot --dry-run --agree-tos --standalone certonly --register-unsafely-without-email
    --pre-hook 'systemctl stop dnsdist'
    --post-hook /usr/local/bin/
    -d {{ DOMAIN }} -d doh.{{ DOMAIN }} -d dot.{{ DOMAIN }}

# vim: sts=2 sw=2 ts=2 et

chmod +x /usr/local/bin/


Now open your firewall to the below TCP Ports:

ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 853/tcp
  • TCP 80 for certbot
  • TCP 443 for dnsdist (DoT) and certbot !
  • TCP 853 for dnsdist (DoH)

Let’s Encrypt

When you are ready, run the script


That’s it !


For this blog post, my test settings are:


DoT - Client

From systemd 243+ there is an option to validate certificates on DoT but

systemd-resolved only validates the DNS server certificate if it is issued for the server’s IP address (a rare occurrence).

so it is best to use: opportunistic


systemctl restart systemd-resolved


resolvectl query                    -- link: eth0

-- Information acquired via protocol DNS in 1.9ms.
-- Data is authenticated: no

DoH - Client

Firefox Settings


Firefox TRR



Click on DNS leak test site to verify


Tag(s): DoH, DoT, PowerDNS, dnsdist

Pitfalls for OMEMO Implementations – Part 1: Inactive Devices

Smack’s OMEMO implementation received a security audit a while ago (huge thanks to the Guardian Project for providing the funding!). Radically Open Security, a non-profit pentesting group from the Netherlands focused on free software and ethical hacking went through the code in great detail to check its correctness and to search for any vulnerabilities. In the end they made some findings, although I wouldn’t consider them catastrophically bad (full disclosure – its my code, so I might be biased :D). In this post I want to go over two of the finding and discuss, what went wrong and how the issue was fixed.

Finding 001: Inactive Devices break Forward Secrecy

Inactive devices which no longer come online retain old chaining keys, which, if compromised, break
confidentiality of all messages from that key onwards.

Penetration Test Report – Radically Open Security
  • Vulnerability Type: Loss of Forward Secrecy.
  • Threat Level: High

Finding 003: Read-Only Devices Compromise Forward Secrecy

Read-only devices never update their keys and thus forfeit forward security.

Penetration Test Report – Radically Open Security
  • Vulnerability Type: Loss of Forward Secrecy.
  • Threat Level: Elevated

These vulnerabilities have the same cause and can be fixed by the same patch. They can happen, if a contact has a device which doesn’t come online (Finding 001), or just doesn’t respond to messages (Finding 003) for an extended period of time. To understand the issue, we have to look deeper into how the cryptography behind the double ratchet works.


In OMEMO, your device has a separate session with every device it sends messages to. In case of one-to-one chat, your phone may have a session with your computer (you can read messages you sent from your phone on your computer too), with your contacts phone, your contacts computer and so on.

In each session, the cryptography happens in a kind of a ping pong way. Lets say you send two messages, then your contact responds with 4 messages, then its your turn again to send some messages and so on and so forth.

To better illustrate whats happening behind the scenes in the OMEMO protocol, lets compare an OMEMO chat with a discussion which is moderated using a “talking stick” (or “speachers staff”). Every time you want to send a message, you acquire the talking stick. Now you can send as many messages as you want. If another person wants to send you a message, you’ve got to hand them the stick first.

Now, cryptographically OMEMO is based on the double ratchet. As the name suggests, the double ratchet consists of two ratchets; the Diffie-Hellman-Ratchet and the Symmetric-Key-Ratchet.

The Symmetric-Key-Ratchet is responsible to encrypt every message you send with a new key. Roughly speaking, this is done by deriving a fresh encryption key from the key which was used to encrypt the previous message you sent. This procedure is called Key Derivation Function Chain (KDF-Chain). The benefit of this technique is, that if an attacker manages to break the key of a message, they cannot use that key to decrypt any previous messages (this is called “forward secrecy“). They can however use that key to derive the message keys of following messages, which is bad.

Functional Principle of a Key Derivation Function Chain

This is where the Diffie-Hellman-Ratchet comes into play. It replaces the symmetric ratchet key every time the “talking stick” gets handed over. Whenever you pass it to your contact (by them sending a message to you), your device executes a Diffie-Hellman key exchange with your contact and the result is used to generate a new key for the symmetric ratchet. The consequence is, that whenever the talking stick gets handed over, the attacker can no longer decrypt any following messages. This is why the protocol is described as “self healing”. The cryptographic term is called “future secrecy“.

Functional Principle of the Diffie-Hellman Ratchet Algorithm


So, what’s wrong with this technique?

Well, lets say you have a device that you used only once or twice with OMEMO and which is now laying in the drawer, collecting dust. That device might still have active sessions with other devices. The problem is, that it will never acquire the talking stick (it will never send a message). Therefore the Diffie-Hellman-Ratchet will never be forwarded, which will lead to the base key of the Symmetric-Key-Ratchet never being replaced. The consequence is, that once an attacker manages to break one key of the chain, they can use that key to derive all following keys, which will get them access to all messages that device receives in that session. The property of “forward secrecy” is lost.

Smack-omemo already kind of dealt with that issue by removing inactive devices from the device list after a certain amount of time (two weeks). This does however only include own devices, not inactive devices of a contact. In the past I tried to fix this by ignoring inactive devices of a contact, by refusing to encrypt message for them. However, this lead to deadlocks, where two devices were mutually ignoring each other, resulting in a situation where no device could send a message to the other one to recover. As a consequence, Smack omitted deactivating stale contacts devices, which resulted in the vulnerability.

The Patch

This problem could not really be solved without changing the XEP (I’d argue). In my opinion the XEP must specify an explicit way of dealing with inactive devices. Therefore I proposed to use message counters, which gets rid of the deadlock problem. A device is considered as read-only, if the other party sent n messages without getting a reply. That way we can guarantee, that in a worst case scenario, an attacker can get access to n messages. That magical number n would need to be determined though.
I opened a pull request against XEP-0384 which introduces these changes.

Mitigation for the issue has been merged into Smack’s source code already đŸ™‚

As another prevention against this vulnerability, clients could send empty ping messages to forward the Diffie-Hellman-Ratchet. This does however require devices to take action and does not work with clients that are permanently offline though. Both changes would however go hand in hand to improve the OMEMO user experience and security. Smack-omemo already offers the client developer a method to send ping messages đŸ™‚

I hope I could help any fellow developers to avoid this pitfall and spark some discussion around how to address this issue in the XEP.

Happy (ethical) Hacking!

Sunday, 08 December 2019

Kubernetes as a Service with Rancer2 at Hetzner using Terraform and Helm

In this blog post you will find my personal notes on how to setup a Kubernetes as a Service (KaaS). I will be using Terraform to create the infrastructure on Hetzner’s VMs, Rancher for KaaS and Helm to install the first application on Kubernetes.


Many thanks to dear friend: adamo for his help.


Let’s build our infrastructure!
We are going to use terraform to build 5 VMs

  • One (1) master
  • One (1) etcd
  • Two (2) workers
  • One (1) for the Web dashboard

I will not go to much details about terraform, but to have a basic idea

provider "hcloud" {
    token = var.hcloud_token

data "template_file" "userdata" {
  template = "${file("user-data.yml")}"
  vars = {
    hostname = var.domain
    sshdport = var.ssh_port

resource "hcloud_server" "node" {
  count       = 5
  name        = "rke-${count.index}"
  image       = "ubuntu-18.04"
  server_type = "cx11"
  user_data   = data.template_file.userdata.rendered

output "IPv4" {
  value = hcloud_server.node.*.ipv4_address

In my user-data (cloud-init) template, the most important lines are these

  - usermod -a -G docker deploy
  - ufw allow 6443/tcp
  - ufw allow 2379/tcp
  - ufw allow 2380/tcp
  - ufw allow 80/tcp
  - ufw allow 443/tcp

build infra

$ terraform init
$ terraform plan
$ terraform apply


IPv4 = [

In the end we will see something like this on hetzner cloud

hetzner VMs

Rancher Kubernetes Engine

Take a look here for more details about what is required and important on using rke: Requirements.

We are going to use the rke aka the Rancher Kubernetes Engine, an extremely simple, lightning fast Kubernetes installer that works everywhere.


Download the latest binary from github:
Release Release v1.0.0

$ curl -sLO
$ chmod +x rke_linux-amd64
$ sudo mv rke_linux-amd64 /usr/local/bin/rke


$ rke --version

rke version v1.0.0

rke config

We are ready to configure our Kubernetes Infrastructure using the first 4 VMs.

$ rke config


[+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]:
[+] Number of Hosts [1]: 4
[+] SSH Address of host (1) [none]: 78.47.6x.yyy
[+] SSH Port of host (1) [22]:
[+] SSH Private Key Path of host (78.47.6x.yyy) [none]:
[-] You have entered empty SSH key path, trying fetch from SSH key parameter
[+] SSH Private Key of host (78.47.6x.yyy) [none]:
[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~/.ssh/id_rsa
[+] SSH User of host (78.47.6x.yyy) [ubuntu]:
[+] Is host (78.47.6x.yyy) a Control Plane host (y/n)? [y]:
[+] Is host (78.47.6x.yyy) a Worker host (y/n)? [n]: n
[+] Is host (78.47.6x.yyy) an etcd host (y/n)? [n]: n
[+] Override Hostname of host (78.47.6x.yyy) [none]: rke-master
[+] Internal IP of host (78.47.6x.yyy) [none]:
[+] Docker socket path on host (78.47.6x.yyy) [/var/run/docker.sock]: 


[+] SSH Address of host (2) [none]: 78.47.1x.yyy
[+] SSH Port of host (2) [22]:
[+] SSH Private Key Path of host (78.47.1x.yyy) [none]:
[-] You have entered empty SSH key path, trying fetch from SSH key parameter
[+] SSH Private Key of host (78.47.1x.yyy) [none]:
[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~/.ssh/id_rsa
[+] SSH User of host (78.47.1x.yyy) [ubuntu]:
[+] Is host (78.47.1x.yyy) a Control Plane host (y/n)? [y]: n
[+] Is host (78.47.1x.yyy) a Worker host (y/n)? [n]: n
[+] Is host (78.47.1x.yyy) an etcd host (y/n)? [n]: y
[+] Override Hostname of host (78.47.1x.yyy) [none]: rke-etcd
[+] Internal IP of host (78.47.1x.yyy) [none]:
[+] Docker socket path on host (78.47.1x.yyy) [/var/run/docker.sock]: 



[+] SSH Address of host (3) [none]: 78.46.2x.yyy
[+] SSH Port of host (3) [22]:
[+] SSH Private Key Path of host (78.46.2x.yyy) [none]:
[-] You have entered empty SSH key path, trying fetch from SSH key parameter
[+] SSH Private Key of host (78.46.2x.yyy) [none]:
[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~/.ssh/id_rsa
[+] SSH User of host (78.46.2x.yyy) [ubuntu]:
[+] Is host (78.46.2x.yyy) a Control Plane host (y/n)? [y]: n
[+] Is host (78.46.2x.yyy) a Worker host (y/n)? [n]: y
[+] Is host (78.46.2x.yyy) an etcd host (y/n)? [n]: n
[+] Override Hostname of host (78.46.2x.yyy) [none]: rke-worker-01
[+] Internal IP of host (78.46.2x.yyy) [none]:
[+] Docker socket path on host (78.46.2x.yyy) [/var/run/docker.sock]: 


[+] SSH Address of host (4) [none]: 78.47.4x.yyy
[+] SSH Port of host (4) [22]:
[+] SSH Private Key Path of host (78.47.4x.yyy) [none]:
[-] You have entered empty SSH key path, trying fetch from SSH key parameter
[+] SSH Private Key of host (78.47.4x.yyy) [none]:
[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~/.ssh/id_rsa
[+] SSH User of host (78.47.4x.yyy) [ubuntu]:
[+] Is host (78.47.4x.yyy) a Control Plane host (y/n)? [y]: n
[+] Is host (78.47.4x.yyy) a Worker host (y/n)? [n]: y
[+] Is host (78.47.4x.yyy) an etcd host (y/n)? [n]: n
[+] Override Hostname of host (78.47.4x.yyy) [none]: rke-worker-02
[+] Internal IP of host (78.47.4x.yyy) [none]:
[+] Docker socket path on host (78.47.4x.yyy) [/var/run/docker.sock]: 

Network Plugin Type

[+] Network Plugin Type (flannel, calico, weave, canal) [canal]: 


[+] Authentication Strategy [x509]:
[+] Authorization Mode (rbac, none) [rbac]: none
[+] Kubernetes Docker image [rancher/hyperkube:v1.16.3-rancher1]:
[+] Cluster domain [cluster.local]:
[+] Service Cluster IP Range []:
[+] Enable PodSecurityPolicy [n]:
[+] Cluster Network CIDR []:
[+] Cluster DNS Service IP []:
[+] Add addon manifest URLs or YAML files [no]: 


the rke config will produce a cluster yaml file, for us to review or edit in case of misconfigure

$ ls -l cluster.yml
-rw-r----- 1 ebal ebal 4720 Dec  7 20:57 cluster.yml

rke up

We are ready to setup our KaaS by running:

$ rke up
INFO[0000] Running RKE version: v1.0.0
INFO[0000] Initiating Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [78.47.6x.yyy]
INFO[0000] [dialer] Setup tunnel for host [78.47.1x.yyy]
INFO[0000] [dialer] Setup tunnel for host [78.46.2x.yyy]
INFO[0000] [dialer] Setup tunnel for host [78.47.7x.yyy]
INFO[0329] [dns] DNS provider coredns deployed successfully
INFO[0329] [addons] Setting up Metrics Server
INFO[0329] [addons] Saving ConfigMap for addon rke-metrics-addon to Kubernetes
INFO[0329] [addons] Successfully saved ConfigMap for addon rke-metrics-addon to Kubernetes
INFO[0329] [addons] Executing deploy job rke-metrics-addon
INFO[0335] [addons] Metrics Server deployed successfully
INFO[0335] [ingress] Setting up nginx ingress controller
INFO[0335] [addons] Saving ConfigMap for addon rke-ingress-controller to Kubernetes
INFO[0335] [addons] Successfully saved ConfigMap for addon rke-ingress-controller to Kubernetes
INFO[0335] [addons] Executing deploy job rke-ingress-controller
INFO[0341] [ingress] ingress controller nginx deployed successfully
INFO[0341] [addons] Setting up user addons
INFO[0341] [addons] no user addons defined
INFO[0341] Finished building Kubernetes cluster successfully 


The output of rke will produce a local kube config cluster yaml file for us to connect to kubernetes cluster.


Let’s test our k8s !

$ kubectl --kubeconfig=kube_config_cluster.yml get nodes -A
NAME           STATUS   ROLES          AGE    VERSION
rke-etcd       Ready    etcd           2m5s   v1.16.3
rke-master     Ready    controlplane   2m6s   v1.16.3
rke-worker-1   Ready    worker         2m4s   v1.16.3
rke-worker-2   Ready    worker         2m2s   v1.16.3

$ kubectl --kubeconfig=kube_config_cluster.yml get pods -A
NAMESPACE       NAME                                      READY   STATUS      RESTARTS   AGE
ingress-nginx   default-http-backend-67cf578fc4-nlbb6     1/1     Running     0          96s
ingress-nginx   nginx-ingress-controller-7scft            1/1     Running     0          96s
ingress-nginx   nginx-ingress-controller-8bmmm            1/1     Running     0          96s
kube-system     canal-4x58t                               2/2     Running     0          114s
kube-system     canal-fbr2w                               2/2     Running     0          114s
kube-system     canal-lhz4x                               2/2     Running     1          114s
kube-system     canal-sffwm                               2/2     Running     0          114s
kube-system     coredns-57dc77df8f-9h648                  1/1     Running     0          24s
kube-system     coredns-57dc77df8f-pmtvk                  1/1     Running     0          107s
kube-system     coredns-autoscaler-7774bdbd85-qhs9g       1/1     Running     0          106s
kube-system     metrics-server-64f6dffb84-txglk           1/1     Running     0          101s
kube-system     rke-coredns-addon-deploy-job-9dhlx        0/1     Completed   0          110s
kube-system     rke-ingress-controller-deploy-job-jq679   0/1     Completed   0          98s
kube-system     rke-metrics-addon-deploy-job-nrpjm        0/1     Completed   0          104s
kube-system     rke-network-plugin-deploy-job-x7rt9       0/1     Completed   0          117s

$ kubectl --kubeconfig=kube_config_cluster.yml get componentstatus
NAME                 AGE
controller-manager   <unknown>
scheduler            <unknown>
etcd-0               <unknown>             <unknown>

$ kubectl --kubeconfig=kube_config_cluster.yml get deployments -A
ingress-nginx   default-http-backend   1/1     1            1           2m58s
kube-system     coredns                2/2     2            2           3m9s
kube-system     coredns-autoscaler     1/1     1            1           3m8s
kube-system     metrics-server         1/1     1            1           3m4s

$ kubectl --kubeconfig=kube_config_cluster.yml get ns
NAME              STATUS   AGE
default           Active   4m28s
ingress-nginx     Active   3m24s
kube-node-lease   Active   4m29s
kube-public       Active   4m29s
kube-system       Active   4m29s


Now login to the 5th VM we have in Hetzner:

ssh "78.47.4x.yyy" -l ubuntu -p zzzz

and install the stable version of Rancher2

$ docker run -d
    -p 80:80 -p 443:443
    --name rancher2
    -v /opt/rancher:/var/lib/rancher

Caveat: I have create a domain and assigned to this hostname the IP of the latest VMs!
Now I can use letsencrypt with rancher via acme-domain.


$ docker images -a
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rancher/rancher     stable              5ebba94410d8        10 days ago         654MB

$ docker ps -a -a
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                      NAMES
8f798fb8184c        rancher/rancher:stable   " --acm…"   17 seconds ago      Up 15 seconds>80/tcp,>443/tcp   rancher2


Before we continue, we need to give access to these VMs so they can communicate with each other. In cloud you can create a VPC with the correct security groups. But with VMs the easiest way is to do something like this:

sudo ufw allow from "78.47.6x.yyy",
sudo ufw allow from "78.47.1x.yyy",
sudo ufw allow from "78.46.2x.yyy",
sudo ufw allow from "78.47.7x.yyy",
sudo ufw allow from "78.47.4x.yyy",


Open your browser and type the IP of your rancher2 VM:


or (in my case):

and follow the below instructions







Connect cluster with Rancher2

Download the racnher2 yaml file to your local directory:

$ curl -sLo rancher2.yaml

And apply this yaml file to your kubernetes cluster:

$ kubectl --kubeconfig=kube_config_cluster.yml apply -f rancher2.yaml unchanged unchanged
namespace/cattle-system unchanged
serviceaccount/cattle unchanged unchanged
secret/cattle-credentials-2704c5f created configured
deployment.apps/cattle-cluster-agent configured
daemonset.apps/cattle-node-agent configured

Web Gui



kubectl config

We can now use the Rancher kubectl config by downloading from here:


In this post, it is rancher2.config.yml


Final step is to use helm to install an application to our kubernetes cluster

download and install

$ curl -sfL | tar -zxf -

$ chmod +x linux-amd64/helm
$ sudo mv linux-amd64/helm /usr/local/bin/

Add Repo

$ helm repo add stable
"stable" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈ 


Install weave scope to rancher:

$ helm --kubeconfig rancher2.config.yml install stable/weave-scope --generate-name
NAME: weave-scope-1575800948
LAST DEPLOYED: Sun Dec  8 12:29:12 2019
NAMESPACE: default
STATUS: deployed
You should now be able to access the Scope frontend in your web browser, by
using kubectl port-forward:

kubectl -n default port-forward $(kubectl -n default get endpoints
weave-scope-1575800948-weave-scope -o jsonpath='{.subsets[0].addresses[0]}') 8080:4040

then browsing to http://localhost:8080/.
For more details on using Weave Scope, see the Weave Scope documentation:


Last, we are going to use kubectl to create a forwarder

$ kubectl --kubeconfig=rancher2.config.yml -n default port-forward $(kubectl --kubeconfig=rancher2.config.yml -n default get endpoints weave-scope-1575800948-weave-scope -o jsonpath='{.subsets[0].addresses[0]}') 8080:4040
Forwarding from -> 4040
Forwarding from [::1]:8080 -> 4040

Open your browser in this url:



That’s it !

Saturday, 30 November 2019

A beginner's guide to C++ Ranges and Views.

C++ Ranges are one of the major new things in C++20 and “views” are a big part of ranges. This article is a short introduction for programmers that are new to C++ Ranges.


You don’t need to have any prior knowledge of C++ Ranges, but you should have basic knowledge of C++ iterators and you should have heard of C++ Concepts before. There are various resources on C++ Concepts, e.g. Good Concepts, Wikipedia (although both contain slightly outdated syntax).

This article is based on library documentation that I wrote for the SeqAn3 library. The original is available here. There is also beginner’s documentation on C++ Concepts over there.

Since none of the large standard libraries ship C++ Ranges right now, you need to use the range-v3 library if you want to try any of this. If you do, you need to replace the std::ranges:: prefixes with just ranges:: and any std::views:: prefixes with ranges::views::.


Traditionally most generic algorithms in the C++ standard library, like std::sort, take a pair of iterators (e.g. the object returned by begin()). If you want to sort a std::vector v, you have to call std::sort(v.begin(), v.end()) and not std::sort(v). Why was this design with iterators chosen? It is more flexible, because it allows e.g.:

  • sorting only all elements after the fifth one:

    1 std::sort(v.begin() + 5, v.end())
  • using non-standard iterators like reverse iterators (sorts in reverse order):

    1 std::sort(v.rbegin(), v.rend());
  • combine both (sorts all elements except the last 5 in reverse order):

    1 std::sort(v.rbegin() + 5, v.rend());

But this interface is less intuitive than just calling std::sort on the entity that you wish to sort and it allows for more mistakes, e.g. mixing two incompatible iterators. C++20 introduces the notion of ranges and provides algorithms that accept such in the namespace std::ranges::, e.g. std::ranges::sort(v) now works if v is range – and vectors are ranges!

What about the examples that suggest superiority of the iterator-based approach? In C++20 you can do the following:

  • sorting only all elements after the fifth one:

    1 std::ranges::sort(std::views::drop(v, 5));
  • sorting in reverse order:

    1 std::ranges::sort(std::views::reverse(v));
  • combine both:

    1 std::ranges::sort(std::views::drop(std::views::reverse(v), 5));

We will discuss later what std::views::reverse(v) does, for now it is enough to understand that it returns something that appears like a container and that std::ranges::sort can sort it. Later you will see that this approach offers even more flexibility than working with iterators.


Ranges are an abstraction of “a collection of items”, or “something iterable”. The most basic definition requires only the existence of begin() and end() on the range.

Range concepts

There are different ways to classify ranges, the most important one is by the capabilities of its iterator.

Ranges are typically input ranges (they can be read from), output ranges (they can be written to) or both. E.g. a std::vector<int> is both, but a std::vector<int> const would only be an input range.

Input ranges have different strengths that are realised through more refined concepts (i.e. types that model a stronger concept, always also model the weaker one):

Concept Description
std::ranges::input_range can be iterated from beginning to end at least once
std::ranges::forward_range can be iterated from beginning to end multiple times
std::ranges::bidirectional_range iterator can also move backwards with --
std::ranges::random_access_range you can jump to elements in constant-time []
std::ranges::contiguous_range elements are always stored consecutively in memory

These concepts are derived directly from the respective concepts on the iterators, i.e. if the iterator of a range models std::forward_iterator, than the range is a std::ranges::forward_range.

For the well-known containers from the standard library this matrix shows which concepts they model:

std::forward_list std::list std::deque std::array std::vector

There are also range concepts that are independent of input or output or one of the above concepts, e.g. std::ranges::sized_range which requires that the size of a range is retrievable by std::ranges::size() (in constant time).

Storage behaviour

Containers are the ranges most well known, they own their elements. The standard library already provides many containers, see above.

Views are ranges that are usually defined on another range and transform the underlying range via some algorithm or operation. Views do not own any data beyond their algorithm and the time it takes to construct, destruct or copy them should not depend on the number of elements they represent. The algorithm is required to be lazy-evaluated so it is feasible to combine multiple views. More on this below.

The storage behaviour is orthogonal to the range concepts defined by the iterators mentioned above, i.e. you can have a container that satisfies std::ranges::random_access_range (e.g. std::vector does, but std::list does not) and you can have views that do so or don’t.



A key feature of views is that whatever transformation they apply, they do so at the moment you request an element, not when the view is created.

1 std::vector vec{1, 2, 3, 4, 5, 6};
2 auto v = std::views::reverse(vec);

Here v is a view; creating it neither changes vec, nor does v store any elements. The time it takes to construct v and its size in memory is independent of the size of vec.

1 std::vector vec{1, 2, 3, 4, 5, 6};
2 auto v = std::views::reverse(vec);
3 std::cout << *v.begin() << '\n';

This will print “6”, but the important thing is that resolving the first element of v to the last element of vec happens on-demand. This guarantees that views can be used as flexibly as iterators, but it also means that if the view performs an expensive transformation, it will have to do so repeatedly if the same element is requested multiple times.


You may have wondered why I wrote

1 auto v = std::views::reverse(vec);

and not

1 std::views::reverse v{vec};

That’s because std::views::reverse is not the view itself, it’s an adaptor that takes the underlying range (in our case the vector) and returns a view object over the vector. The exact type of this view is hidden behind the auto statement. This has the advantage, that we don’t need to worry about the template arguments of the view type, but more importantly the adaptor has an additional feature: it can be chained with other adaptors!

1 std::vector vec{1, 2, 3, 4, 5, 6};
2 auto v = vec | std::views::reverse | std::views::drop(2);
4 std::cout << *v.begin() << '\n';

What will this print?

Here is the solution It will print “4”, because “4” is the 0-th element of the reversed string after dropping the first two.

In the above example the vector is “piped” (similar to the unix command line) into the reverse adaptor and then into the drop adaptor and a combined view object is returned. The pipe is just a different notation that improves readability, i.e. vec | foo | bar(3) | baz(7) is equivalent to baz(bar(foo(vec), 3), 7). Note that accessing the 0th element of the view is still lazy, determining which element it maps to happens at the time of access.


Create a view on std::vector vec{1, 2, 3, 4, 5, 6}; that filters out all uneven numbers and squares the remaining (even) values, i.e.

1 std::vector vec{1, 2, 3, 4, 5, 6};
2 auto v = vec | // ...?
4 std::cout << *v.begin() << '\n'; // should print 4

To solve this you can use std::views::transform and std::views::filter. Both take a invocable as argument, e.g. a lambda expression. std::views::transform applies the lambda on each element in the underlying range and std::views::filter “removes” those elements that its lambda function evaluates to false for.

Here is the solution

1 std::vector vec{1, 2, 3, 4, 5, 6};
2 auto v = vec
3        | std::views::filter(   [] (auto const i) { return i % 2 == 0; })
4        | std::views::transform([] (auto const i) { return i*i; });
6 std::cout << *v.begin() << '\n'; // prints 4

View concepts

Views are a specific kind of range that is formalised in the std::ranges::view concept. Every view returned by a view adaptor models this concept, but which other range concepts are modeled by a view?

It depends on the underlying range and also the view itself. With few exceptions, views don’t model more/stronger range concepts than their underlying range (except that they are always a std::ranges::view) and they try to preserve as much of the underlying range’s concepts as possible. For instance the view returned by std::views::reverse models std::ranges::random_access_range (and weaker concepts) iff the underlying range also models the respective concept. It never models std::ranges::contiguous_range, because the third element of the view is not located immediately after the second in memory (but instead before the second).

Perhaps surprising to some, many views also model std::ranges::output_range if the underlying range does, i.e. views are not read-only:

1 std::vector vec{1, 2, 3, 4, 5, 6};
2 auto v = vec | std::views::reverse | std::views::drop(2);
4 *v.begin() = 42; // now vec == {1, 2, 3, 42, 5, 6 } !!


Have a look at the solution to the previous exercise (filter+transform). Which of the following concepts do you think v models?

Concept yes/no?

Here is the solution

Concept yes/no?

The filter does not preserve random-access and therefore not contiguity, because it doesn’t “know” which element of the underlying range is the i-th one in constant time. It cannot “jump” there, it needs to move through the underlying range element-by-element. This also means we don’t know the size.

The transform view would be able to jump, because it always performs the same operation on every element independently of each other; and it would also preserve sized-ness because the size remains the same. In any case, both properties are lost due to the filter. On the other hand the transform view produces a new element on every access (the result of the multiplication), therefore v is not an output range, you cannot assign values to its elements. This would have prevented modelling contiguous-range as well – if it hadn’t been already by the filter – because values are created on-demand and are not stored in memory at all.

Understanding which range concepts “survive” which particular view needs some practice. For the SeqAn3 library we try to document this in detail, I hope we will see something similar on

Post scriptum

I am quite busy currently with my PhD thesis, but I plan to publish some smaller articles on ranges and views before the holiday season. Most will be based on text pieces I have already written but that never found their way to this blog (library documentation, WG21 papers, snippets from the thesis, …).

Thanks for reading, I hope this article was helpful! If you have any questions, please comment here or on twitter/mastodon.

Friday, 22 November 2019

Align user IDs inside and outside Docker with subuser mapping

  • Seravo
  • 16:34, Friday, 22 November 2019

While Docker is quite handy in many ways, one inconvenient aspect is that if one mounts some host machine directory inside Docker, and the Docker image does something to those files as a non-root user, one will run into problems if the UID of the host machine user and the Docker image user do not match.

Consider this scenario: I my user ’otto’ is running on the host machine as UID 1001. Then I spin up a development image with Docker where I do development and store files in a locally mounted directory. Inside there is a temporary Docker user called ’vagrant’ with the UID 1000. Now whenever any files are updated (say via composer update for example) it will fail to work, as the mounted directory is owned by UID 1001 and the Docker image user with UID 1000 will not be able to write anything.

Docker run –user does not help

If one has complete control of the Docker image, one could modify the UID of the Docker user and rebuild. Most of the time that is however not an option. Docker run has the option --user but that does not help if the Docker image already has created the user with a particular name and UID/GID settings.

Subuser and subgroup mapping to the rescue

To use subuser and subgroup mapping, first enable the feature in Docker by editing the /etc/docker/daemon.json config file to include:

  "userns-remap": "otto"

Naturally, replace ’otto’ with your own username. For changes to take effect, restart Docker with sudo systemctl restart docker and check with sudo systemctl status docker that is restarted wihtout failures and is running.

To instruct the mapping, edit the files /etc/subuid (for user IDs) and /etc/subgid (for group IDs). In the case above, I mapped users 0-1000 to have the same UID/GID both inside and outside Docker. I want for root to have UID 0 all the time. However the user IDs should start from 1001 and not from 1000. This way the most common UID inside the Docker container 1000 will match my own laptop UID 1001 of my user.



Read more

Original idea by

Details at and

Sunday, 17 November 2019

New challenges for Free Software business models

This year the FSFE community meeting was combined with the “South Tyrol Free Software Conference” (SFScon) in Bolzano. For me this was a special event because the first international FSFE community meeting ever happened as well at the SFScon in 2006. Back then I met many people from FSFE in person for the first time. For me this was the starting point for getting more and more involved in the Free Software Foundation Europe.

At this years conference I gave a talk about the “New challenges for Free Software business models” at the FSFE track. A few weeks ago I published a article about this topic in the German Linux Magazine. As many of you may know, Free Software as such is not a business model but a license model which can be combined with many different business and development models.

Distinction between business-, license- and development-model

I’m convinced that working business models around Free Software are a important building block for Free Software to compete successfully with the proprietary software world. The questions how to make money with Free Software and how to build sustainable and strong companies around Free Software are important topics almost right from the beginning of the Free Software movement. Over time we come up with various business models which worked quite well. But the change in technology over the last few years start to put some of the more successful business models at risk. The talk summarized the current challenges and invited the audience to think about possible solutions.

(This blog contain some presentation slides, you can see them here.)

The recording can be found here.

After the talk I had many interesting discussions. Most people agreed that this is a problem. One suggestion was to have a look at Mozilla for a successful business model. Another idea was that the big IaaS providers might buy some of the companies behind the software in the future and continue the development, which wouldn’t be a problem as long as they would stick to Free Software. Yet another interesting thought was that if you look at the software market as a whole you will realize that Free Software is still a small piece of the cake. As long as the cake as a whole and the Free Software part in particular grows fast we don’t have to worry that much how the Free Software part is split up, there will be enough space for everyone. The was also mentioned as a possible example for a successful business model and many more ideas floated around.

I don’t want to comment on the individual ideas here but it shows that we had a lively discussion with many interesting ideas and thoughts.

Do you also have some thoughts around this topic? Feel free to share them in the comments!

Sunday, 10 November 2019

19.12 releases branches created

The branch naming has changed to try to accommodate for the stopping of the "KDE Applications" brand, it's now called

Make sure you commit anything you want to end up in the 19.12 releases to them

We're already past the dependency freeze.

The Freeze and Beta is this Thursday 14 of November.

More interesting dates
November 28, 2019: KDE Applications 19.12 RC (19.11.90) Tagging and Release
December 5, 2019: KDE Applications 19.12 Tagging
December 12, 2019: KDE Applications 19.12 Release


P.S: Yes, this release unfortunately falls in the middle of the debranding of "KDE Applications" and there's still a few things called "KDE Applications" here and there

[*] There's a small issue with kwave we're working on figuring it out

Thursday, 07 November 2019

You can use a C++11 range for loop over a static array

void DeviceListing::populateListing(const show showStatus)
const Solid::DeviceInterface::Type needHardware[] = {


for (unsigned int i = 0; i < (sizeof(needHardware)/sizeof(Solid::DeviceInterface::Type)); i++) {
QTreeWidgetItem *tmpDevice = createListItems(needHardware[i]);
deviceMap[needHardware[i]] = static_cast(tmpDevice);

if ((tmpDevice->childCount() > 0) || (showStatus == ALL)) {

in C++11 you can rewrite it in the much easier to read

void DeviceListing::populateListing(const show showStatus)
const Solid::DeviceInterface::Type needHardware[] = {


for (const Solid::DeviceInterface::Type nh : needHardware) {
QTreeWidgetItem *tmpDevice = createListItems(nh);
deviceMap[nh] = static_cast(tmpDevice);

if ((tmpDevice->childCount() > 0) || (showStatus == ALL)) {

Sunday, 03 November 2019

On “Clean Architecture”

I recently did what I rarely do: buy and read an educational book. Shocking, I know, but I can assure you that I’m fine đŸ˜‰

The book I ordered is Clean Architecture – A Craftsman’s Guide to Software Structure and Design by Robert C. Martin. As the title suggests it is about software architecture.

I’ve barely read half of the book, but I’ve already learned a ton! I find it curious that as a halfway decent programmer, I often more or less know what Martin is talking about when he is describing a certain architectural pattern, but I often didn’t know said pattern had a name, or what consequences using said pattern really implied. It is really refreshing to see the bigger picture and having all the pros and cons of certain design decisions layed out in an overview.

One important point the book is trying to put across is how important it is to distinguish between important things like business rule, and not so important things as details. Let me try to give you an example.

Lets say I want to build a reactive Android XMPP chat application using Smack (foreshadowing? đŸ˜‰ ). Lets identify the details. Surely Smack is a detail. Even though I’d be using Smack for some of the core functionalities of the app, I could just as well chose another XMPP library like babbler to get the job done. But there are even more details, Android for example.

In fact, when you strip out all the details, you are left with reactive chat application. Even XMPP is a detail! A chat application doesn’t care, what protocol you use to send and receive messages, heck it doesn’t even care if it is run on Android or on any other device (that can run java).

I’m still not quite sure, if the keyword reactive is a detail, as I’d say it is a more a programming paradigm. Details are things that can easily be switched out and/or extended and I don’t think you can easily replace a programming paradigm.

The book does a great job of identifying and describing simple rules that, when applied to a project lead to a cleaner, more structured architecture. All in all it teaches how important software architecture in general is.

There is however one drawback with the book. It constantly wants to make you want to jump straight into your next big project with lots of features, so it is hard to keep reading while all that excited ;P

If you are a software developer – no matter whether you work on small hobby projects or big enterprise products, whether or not you pursue to become a Software Architect – I can only recommend reading this book!

One more thought: If you want to support a free software project, maybe donating books like this is a way to contribute?

Happy Hacking!

Wednesday, 30 October 2019

Akademy Guest Post: Some thoughts regarding discussion culture

First of all I want to say thanks for hosting me on your blog and letting me write this guest post!

I am Florian MĂźller from Germany, working as a Sysadmin at a small tech collective. I have been a KDE user for years, but I am quite new to the KDE community. This years Akademy was my first and I am looking forward to dive in deeper and contribute more to KDE.

I had a really good time at Akademy in Milano, got to know so many cool people and learned a lot about the KDE community and KDE in general. Thanks a lot to everyone organizing this unique event! I am sure it won't be the last time I'll attend Akademy.

I want to share some things I noticed about discussion culture, please feel free to criticize/discuss/comment, so maybe we can work out some guidelines for that in the future.
First of all some words about discussion culture in general. Because all people are different, we need technics to allow everyone to participate in group discussions. When people are not that confident in general, they tend to not raise their voices in bigger groups, also if they have opponent opinions to the main stream. So in order to have a greater culmination of creativity and ideas, we need to create an atmosphere to make it easier for everyone to participate.

One thing would be the setup of the room the discussion will be held in. Do you want to have a central speaker in the front leading the discussion or do you want to setup the chairs and tables in a circle so everyone is on the same level. Sometimes there are mostly lecture halls available, so time to plan in advance and ask for a different room has to be taken in accountance. The advantage of a circle is that everybody is able to see everyone else. We can also think about forming smaller discussion groups when possible and a topic can be split up into smaller parts.

Another point you can think about is how you organize the turn keeping. One could appoint a person who only deals with turn keeping. We could alternate the job of turn keeping from one discussion to the next. At the beginning of the discussion we should make it transparent on how turn keeping will work. For example we could have a rule that people who want to say something the first time, move to the top of the speakers list to make sure that rare voices will be heard. We could also have a speakers list alternated by gender. There could be a maximum time for speakers so that single people don’t dominate the whole discussion by simply talking others out.

There are many other aspect we can think about when we want to make group discussions more inclusive. With this post, I just wanted to get the discussion started, so we can improve on that in the future.

Here are some links to continue reading:

You can reach me directly at

Sunday, 27 October 2019

My participation in the FSFE 2019 GA meeting in Essen

View on the Ruhr at the venue. Thankful to be among the guests that the FSFE GA invited to join their meeting, I went on 11 October 2019 to the Linuxhotel “Villa Vogelsang” in Essen. Unfortunately it was not possible to buy a separate ticket for the S-bahn on the website of Deutsche Bahn, so […]

Saturday, 26 October 2019

LibreDNS has a new AdBlock endpoint

LibreDNS has a new endpoint

This new endpoint is unique cause it blocks by default Ads & Trackers !



We are currently using Steven Black’s hosts file.


noticeable & mentionable

LibreDNS DOES NOT keep any logs and we are using OpenNIC as TLD Tier1 root NS


Here are my settings


ads doh

Tuesday, 22 October 2019

The 3rd FSFE System Hackers hackathon

On 10 and 11 October, the FSFE System Hackers met in person to tackle problems and new features regarding the servers and services the FSFE is running. The team consists of dedicated volunteers who ensure that the community and staff can work effectively. The recent meeting built on the great work of the past 2 years which have been shaped by large personal and technical changes.

The System Hackers are responsible for the maintenance and development of a large number of services. From the website’s deployment to the mail servers and blogs, from Git to internal services like DNS and monitoring, all these services, virtual machines and physical servers are handled by this friendly group that is always looking forward to welcoming new members.

Overview of the FSFE's services and servers

Overview of the FSFE's services and servers

So in October, six of us met in Cologne. Fittingly, according to a saying in this region, if you do something for the third time, it’s already tradition. So we accomplished this after successful meetings in Berlin (April 2018) and Vienna (March 2019). And although it took place on workdays, it’s been the meeting with the highest participation so far!

Getting. Things. Done!

After the first and second meeting were mostly about getting an overview of historically grown and sparsely documented infrastructure and bringing it into a stable state, we were able to deal with a few more general topics this time. At the same time, we exchanged our knowledge with newly joined team members. Please find the areas we worked on below:

  • Florian migrated the FSFE Blogs to a new server and thereby also updated the underlying Wordpress to the latest version. This has been a major blocker for several other tasks and our largest security risk. There are still a few things left to do, e.g. creating a theme in line with the FSFE design and some announcement to the community. However, the most complicated part is done!
  • Altogether, we upgraded a lot of machines to Debian 10, just after we lifted most servers to Debian 9 in March. Some are still missing, but since the migration is rather painless, we can do that during the next months.
  • We confirmed that the new decentralised backup system setup by myself and based on Borg works fine. This gives us more confidence in our infrastructure.
  • Thanks to Florian and Albert, we finally got rid of the last 2 services that were not using Let’s Encrypt’s self-renewing certificates.
  • Vincent and Francesco took care of finishing the migration of all our Docker containers to use the Docker-in-Docker deployment instead of the hacky Ansible playbooks we used initially. This has a few security advantages and enables the next developments for a more resilient Docker infrastructure.
  • At the moment, all our Docker containers run on one single virtual machine. Although this runs on a Proxmox/Ceph cluster, it’s obviously a single point of failure. However, for a distribution on multiple servers we lack the hardware resources. Nonetheless, we already have concrete plans how to make the Docker setup more resilient as soon as we have more hardware available. Vincent documented this on a wiki page.
  • On the human side, we made sure that all of us know what’s on the plate for the next weeks and months. We have quite a few open issues collected in our Kanban board, and we quickly went through all of them to sketch the possible next steps and distribute responsibilities.

Started projects in the making

Two days are quite some time and we worked hard to use them as effectively as possible, so some tasks have been started but could not be completed – partly because we just did no have enough time, partly because they require more coordination and in-depth discussion:

  • As follow-up on a few unpleasant surprises with Mailman’s default values, we figured that it is important to have an automatic overview of the most sensible settings of the 127 (!) mailing lists we host. Vincent started to work on a way to extract this information in a human- and machine-readable format and merge/compare it with the more verbose documentation on the mailing lists we have internally.
  • Francesco tackled a different weak point we have: monitoring. We lack a tool that informs us immediately about problems in our infrastructure, e.g. defunct core services, full disk drives or expired certificates. Since this is not trivial at all, it requires some more time.
  • Thomas, maintainer of the FSFE wiki, researched on a way to better organise and distribute the SSH accesses in our team. Right now, we have no comfortable way to add or remove SSH keys on our more than 20 machines. His idea is to use an Ansible playbook to manage these, and thereby also create a shared Ansible inventory which can be used as a submodule for the other playbooks we use in the team so we don’t have to maintain all of them individually if a machine is added, changed or removed.
  • One of the most ancient physical machines we still run is hosting the SVN service which is only used by one service now: DNS. We started to work on migrating that over to Git and simultaneously improving the error-checking of the DNS configuration. Albert and I will continue with that gradually.
  • Not on the system hackers meeting itself but two days later, BjĂśrn, Albert and I worked on getting a Nextcloud instance running. Caused by our rather special LDAP setup, we had to debug a lot of strange behaviour but finally figured everything out. Now, the last missing blocker is some user/permission setting within our LDAP. As soon as this is finished, we can shut down one more historically grown, customised-hacked and user-unfriendly service.

Overall, the perspective for the System Hackers is better than ever. We are a growing team carried by motivated and skilled volunteers with a shared vision of how the systems should develop. At the same time, we have a lot of public and internal documentation available to make it easy for new people to join us.

I would like to thank Albert, Florian, Francesco, Thomas and Vincent for their participation in this meeting, and them and all other System Hackers for their dedication to keep the FSFE running!

Tuesday, 15 October 2019

self-hosted Dns Over Https service

LibreOps & LibreDNS

LibreOps announced a new public service: LibreDNS, a new DoH/DoT (DNS over Https/DNS over TLS) free public service for people that want to bypass DNS restrictions and/or want to use TLS in their DNS queries. Firefox has already collaborated with Cloudflare for this case but I believe we can do better than using a centralized public service of a profit-company.

Personal Notes

So here are my personal notes for using LibreDNS in firefox


Open Preferences/Options
firefox options

Enable DoH
firefox doh

TRR mode 2

Now the tricky part.

TRR mode is 2 when you enable DoH. What does this mean?

2 is when firefox is trying to use DoH but if it fails (or timeout) then firefox will go back to ask your operating system’s DNS.

DoH is a URL, so the first time firefox needs to resolve and it will ask your operating system for that.

host file

There is way to exclude from DoH , and use your /etc/hosts file instead your local DNS and enable TRR mode to 3, which means you will ONLY use DoH service for DNS queries.

# grep /etc/hosts

TRR mode 3

and in



DNS Leak

Try DNS Leak Test to verify that your local ISP is NOT your firefox DNS


Thunderbird also supports DoH and here are my settings


PS: Do not forget, this is NOT a global change, just your firefox will ask libredns for any dns query.

Saturday, 12 October 2019

Spotify is Defective by Design

I never used Spotify, since it contains DRM. Instead I still buy DRM-free CDs.
Most of my audio collection is stored in free formats such as FLAC and Ogg Vorbis,
or Red Book in the case of CDs, everything can be played by free players such as
VLC or mpd.

Spotify, which uses a central server, also spies on the listener. Everytime you
listen a song, Spotify knows which song you have listened and when and where.
By contrast free embedded operating systems such as Rockbox do not phone home.
CDs can be baught anonymously and ripped using free software, there is no need
for an internet commection.

Defective by Design recommends the book “Spotify Teardown” which I haven’t read
yet. The book is an innovative investigation of the inner workings of Spotify that
traces the transformation of audio files into streamed experience.

Tuesday, 08 October 2019

2 years since Catalan Independence Referendum, an update

Note 1: This is not KDE or Free Software related, if you're not interested, stop reading, no one is forcing you to read
Note 2: Yes, this is still going to Planet KDE, KDE friends and colleagues ask me about it almost every time we met, so there's definitely interest
Note 3: You're more than welcome to comment, but remember this blog is my house, so don't complain when i don't tolerate stuff i wouldn't tolerate at my home

You may remember Catalonia held an Independence referendum 2 years ago, lots of things have happened since then, I'm going to try to summarize, if you're interested in my initial reaction read my blog from that very same day.

On October 27 2017, following the referendum results, the Parliament of Catalonia declared Independence by a majority of 70 out of 135 MPs. That was mostly ignored by every single country in the world. A few hours later the Spanish government used bigger-army-diplomacy (AKA article 155 of Spanish Constitution) to decide that the Parliament of Catalonia would be suspended and new elections would happen in Catalonia on December 21.

On November 2nd 2017, a judge put most of the Catalan government in jail with the charges of "you've been terribly bad".

They still remain in jail awaiting for trial results (trial finished a few months ago).

Notable exceptions of government officials not in jail are president Carles Puigdemont and Ministers Clara PonsatĂ­ and Toni ComĂ­n, that exiled themselves to other European countries. Spain has tried several times to get European countries to extradite them to Spain because "they've been terribly bad", but that has failed every single time, so they ended up revoking the extradition requests.

Elections happened on December 21 2017, and to shocking surprise of no one, almost virtually the same results happened if you count the pro-independence vs anti-independence blocks.

Since then the Catalan pro-independence government has been basically very low-key in their actions.

Meanwhile, Spain had a its own elections in April this year. They did this nice thing of letting the jailed (but still not sentenced to anything, so innocent) Catalan politicians run, and several of them won Congress seats. Then they said "oh but you know, you're a very dangerous person, so we're not going to let you attend Congress sessions". Not that it matters now, since Spain is unable to govern itself and is having it's 4th election in 4 years this November.

We also had elections in the European Union, and you know what? The same happened! They let catalan-jailed politicians run but then they decided they would not let them take the seats. Actually, this time is even worse since Carles Puigdemont and Toni ComĂ­n, that are living in Brussels without any extradition petition (i.e. they're basically free citizens of Europe), have also been rejected from taking their seats for some esoteric reason.

As a "fun fact", in late 2018 some Spanish regions had elections. Andalucia was one of them and the current government is a coalition of PP+C+VOX, i.e. right wing conservatives, right wing liberals and ultra right wing nut-jobs. One of their first decisions was to put away 100000 euros for grants to teach Spanish to Spanish born people (not for helping immigrants, they're right wing crazies after all) living in Catalonia that don't know how speak Spanish. I'm 99.99% sure the number of people that matches that description is very close to 0 people. You heard well, the poorest region of Spain decided to subsidize the 4th richest region for something that is virtually useless. Thanks!

Much less "fun fact", last week Monday, the Spanish police decided to detain 9 pro-independence people (later to be 7 since 2 were let go) with terrorism charges. The investigation is on-going and technically it should be secret, but we've seen pictures all over the news of what the cops say to be material to make bombs, and all i can see is a pressure cooking pot and some fireworks used typically for Ball de diables.

I don't want to 100% rule out this people having actual plans to do something nasty, but Spanish police/judges/state history of just fabricating charges against people they don't like is so long (An anarchist recently spent *18* months in jail awaiting trial for tweeting stuff "Goku lives, the fight continues" to be then just found innocent after trial) that i would not be surprised either if this is just Spain doing bigger-army-diplomacy again.

TL;DR: Everything is fucked up and I can't really see a way out at this point.

Monday, 07 October 2019

Linux App Summit 2019 schedule is out!

We published the Linux App Summit 2019 schedule last week.

We have a bunch of interesting talks (sadly we had to leave almost 40 almost as interesting talks out, we had lots of awesome submissions!) ranging from flatpak and snaps to how product management is good for Free Software projects with some thought provoking talks thrown in to make us think what's the future of our platform.

I am going and you should totally come too!

The attendance is free but please register at

Better Work Together

Do you want to work on stuff that matters? Since 2010 a group of people in Wellington, New Zealand have been organizing themselves to do just that. Now you can read how they did it!

Their organization is called Enspiral and stories about their amazing journey so far has been documented in the cleverly titled book “Better Work Together“. I have been following Enspiral for years and pretty much devoured the book. It was such a joy to read the personal stories and I am inspired to explore “reinventing Enspiral” (as they call it) here in The Netherlands.

Please go read the book! For free! It is shared under a Creative Commons license, so here is a link to a free and gratis download of the book Better Work Together. Of course, do give some of your money to support the authors if you like what you read.

However, you are probaly super-busy and do not have time to read all the books in the world. So, to help you a bit there the rest of this post is “Better Work Together in Quotes”: a selection of the quotes that resonated a lot with me and that give you an idea of what Enspiral and the book are all about.

If these quotes resonate with you as well, I recommend you to pick up the book and invite you to contact me to see if we can exchange ideas or even join forces to jointly work (more) on stuff that truly matters to us!

Here come the quotes, boldface added to the ones that especially stand out to me:

“Solidarity is not the result of world-changingly good ideas, it is the cause.”

“The work itself – the process of collaboration – ends up as important as whatever product or service is being delivered.”

“If you are sincere in your desire to make the world a better place then your personal success is our number one priority.”

“Thank you for your attention”

– As a species we have never been so powerful.
– With this power we’re changing things.
– We are reaching limits we haven’t encountered before.
– We have the potential (and the urgent need) to change everything.
– What do we do?
– Where do we start?

“Our attention is our power. It is the invisible, constant force that ultimately determines our individual and collective potential.”

“I don’t care what you work on, whether it’s climate change, global poverty, self management, social enterprise, planting trees, gender equality, decolonisation, or steady state economics. If your primary mission is to make the world a better place, your personal success is the reason Enspiral exists.”

“There is a trickle of human energy going into the most important issues of our times. Enspiral exists to help turn that trickle into a river.”

“We organise differently: No one should lead all the time and everyone should lead some of the time.”

“Get paid well to do work you love, with people
you love, while working on a systemic issue you care deeply about.”

“It isn’t easy, but it is possible.”

“The only way to understand Enspiral is to listen to many peo-
ple, which is why many of us are contributing to this book.”

“Either they were working full time on something that wasn’t aligned with their purpose or they were struggling financially as they put all their energy into volunteering.”

“It occurred to me that maybe the best way I could contribute was to help people who wanted to change the world get highly paid contract work.”

“A culture of experimentation”

Managing the cost of failure:
– Respect the status quo
– Copy patterns that work
– Start small
– Let things settle
– Deliver value early

“Distributing leadership”

“We got by, but we never managed to achieve abundance.”

“The dream lives on, continually emerging.”

“We discover the path by walking it together.”

“We tend to follow that paths we can see.”

“Enspiral is agnostic about individual purpose, supporting anyone to do meaningful work – or ‘stuff that matters’ – no matter how they define it. There is no adherence to dogma about specific ways to change the world, other than to help one another.”

“We value the person over the product, the flowering and unfolding of the person over profit. And we’ve created a place where these are not mutually exclusive.”

“Capitalism bombards us with messages of unworthiness and pejorative definitions of success and advertising manipulates our will. Even if we are aware of this, it’s not easy to articulate an alternative story.”

Saturday, 28 September 2019

CentOS 8 NetInstall

a few days ago CentOS-8 (1905) was released and you can find details here ReleaseNotes

Below is a visual guide on how to net-install centos8 1905

notes on a qemu-kvm



Select Language



I have marked the next screens. For netinstall you need to setup first network






Disable kdump


Add Repo


Server Installation





Begin Installation





Make this user administrator










CentOS-8 (1905)


Tag(s): centos8

Thursday, 26 September 2019

Using template file with terraform

When using tf most of times you need to reuse your Infrastructure as Code, and so your code should be written in such way. In my (very simple) use-case, I need to reuse user-data for cloud-init to setup different VMs but I do not want to rewrite basic/common things every time. Luckily, we can use the template_file.


In the below yaml file, you will see that we are using tf string-template to produce hostname with this variable:


here is the file:


disable_root: true
ssh_pwauth: no

  - name: ebal
      - gh:ebal
    shell: /bin/bash

# Set TimeZone
timezone: Europe/Athens

hostname: "${hostname}"

# Install packages
  - mlocate
  - figlet

# Update/Upgrade & Reboot if necessary
package_update: true
package_upgrade: true
package_reboot_if_required: true

# Remove cloud-init
  - figlet "${hostname}" > /etc/motd
  - updatedb


Let’s see our tf variables:

$ cat
variable "hcloud_token" {
    description = "Hetzner Access API token"
    default = ""
variable "gandi_api_token" {
    description = "Gandi API token"
    default = ""
variable "domain" {
    description = " The domain name "
    default = ""

Terraform Template

So we need to use user-data.yml as a template and replace hostname with var.domain

$ cat

Two simple steps:

  • First we read user-data.yml as template and replace hostname with var.domain
  • Then we render the template result to user_data as string
provider "hcloud" {
  token = "${var.hcloud_token}"

data "template_file" "userdata" {
  template = "${file("user-data.yml")}"
  vars = {
    hostname  = "${var.domain}"

resource "hcloud_server" "node1" {
  name = "node1"
  image = "ubuntu-18.04"
  server_type = "cx11"
  user_data = "${data.template_file.userdata.rendered}"
$ terraform version
Terraform v0.12.3

And that’s it !

Tag(s): terraform

Planet FSFE (en): RSS 2.0 | Atom | FOAF |

        Albrechts Blog  Alessandro's blog  Andrea Scarpino's blog  André Ockers on Free Software  Bela's Internship Blog  Bernhard's Blog  Bits from the Basement  Blog of Martin Husovec  Blog – Think. Innovation.  Bobulate  Brian Gough’s Notes  Chris Woolfrey — FSFE UK Team Member  Ciarán’s free software notes  Colors of Noise - Entries tagged planetfsfe  Communicating freely  Daniel Martí's blog  David Boddie - Updates (Full Articles)  ENOWITTYNAME  English Planet – Dreierlei  English on Björn Schießle - I came for the code but stayed for the freedom  English – Alessandro at FSFE  English – Alina Mierlus – Building the Freedom  English – Being Fellow #952 of FSFE  English – Blog  English – FSFE supporters Vienna  English – Free Software for Privacy and Education  English – Free speech is better than free beer  English – Jelle Hermsen  English – Nicolas Jean's FSFE blog  English – Repentinus  English – The Girl Who Wasn't There  English – Thinking out loud  English – Viktor's notes  English – With/in the FSFE  English – gollo's blog  English – mkesper's blog  Escape to freedom  Evaggelos Balaskas - System Engineer  FSFE interviews its Fellows  FSFE – Frederik Gladhorn (fregl)  FSFE – Matej's blog  Fellowship News  Free Software & Digital Rights Noosphere  Free Software with a Female touch  Free Software –  Free Software – Frank Karlitschek_  Free Software – hesa's Weblog  Free as LIBRE  Free, Easy and Others  FreeSoftware – egnun's blog  From Out There  Giacomo Poderi  Green Eggs and Ham  Handhelds, Linux and Heroes  HennR’s FSFE blog  Henri Bergius  Hook’s Humble Homepage  Hugo - FSFE planet  Inductive Bias  Karsten on Free Software  Losca  MHO  Mario Fux  Martin's notes - English  Matthias Kirschner's Web log - fsfe  Max Mehl (English)  Michael Clemens  Myriam's blog  Mäh?  Nice blog  Nico Rikken » fsfe  Nikos Roussos - opensource  Planet FSFE on Iain R. Learmonth  Po angielsku —  Posts - Carmen Bianca Bakker  Posts on Hannes Hauswedell's homepage  Pressreview  Ramblings of a sysadmin (Posts about planet-fsfe)  Rekado  Riccardo (ruphy) Iaconelli – blog  Saint’s Log  Seravo  TSDgeos' blog  Tarin Gamberini  Technology – Intuitionistically Uncertain  The trunk  Thomas Løcke Being Incoherent  Told to blog - Entries tagged fsfe  Tonnerre Lombard  Vitaly Repin. Software engineer's blog  Weblog  Weblog  Weblog  Weblog  Weblog  Weblog  a fellowship ahead  agger's Free Software blog  anna.morris's blog  ayers's blog  bb's blog  blog  en – Florian Snows Blog  en – PB's blog  en – rieper|blog  english – Davide Giunchi  english – Torsten's FSFE blog  foss – vanitasvitae's blog  free software blog  freedom bits  freesoftware – drdanzs blog  fsfe – Thib's Fellowship Blog  julia.e.klein’s blog  marc0s on Free Software  pichel’s blog  planet-en – /var/log/fsfe/flx  polina's blog  softmetz' anglophone Free Software blog  stargrave's blog  tobias_platen's blog  tolld's blog  wkossen’s blog  yahuxo’s blog