Posts about code

GitLab Profile Multi-project Pipeline

I my previous post I relayed setting up a GitLab scheduled profile to add blog posts from this site's RSS feed into my personalised GitLab profile. I discussed setting up a new profile project, creating the README, fetching and parsing the RSS and inserting it into the README, and creating a pipeline to add the changed README back to the profile project. Finally I scheduled the pipeline to run once daily.

Since I don't actually write a post every single day, this mainly runs to no purpose, consuming CI minutes and filling the job logs with pointless runs. It would be better if the pipeline had a job to update the README only after I've actually made a blog post.

Read more…

Personalised GitLab Profile page

On November 22nd, GitLab announced the release of 14.5, in which you can personalise your profile with a README. This README gives you scope to spice up your GitLab profile page quite a bit, for instance you could:

  • Add more details about yourself, beyond what you can fit into GitLab's small biography field
  • Link to other pages, or to projects you would like to feature, turning your profile into a portfolio
  • Include formatting with any of the markup formats which GitLab can understand

This weekend, I decided to turn my personal GitLab profile into something a bit more personalised, and include an automatically updated list of blog posts from this blog's RSS feed.

Read more…

Gone Fishing

This weekend I explored the Fish Shell (a modern interactive Unix shell). I've been dabbling with it for a while, particularly enjoying its interactive features: command suggestions, the TAB-completion menu, the fact that it Just Works™ out of the box, and understands commands and options by groking their man-pages.

I've wanted to port a few of my bash functions which I've been missing: my ssh-pass to load SSH keys onto the Agent's keyring with passphrase supplied by pass, and an alias to get my 1password passphrase from pass as well. Also some directory navigation shortcuts that I've used since my university days. I figured these would be a good introduction to how fish's scripting works, and I was right

Read more…

Python Virtualenv wrapper functions


UPDATE 2020-04-16 This is not how you should do it

These instructions and tools are great if you're going to use Python 2 and Virtualenv. But Python 2 is dead, and there are better ways to do this in Python 3. I'm still exploring the best ways, but I am leaning towards using pyenv + pyenv-virtualenv (or python venvs, or maybe pipenv?). Pyenv takes inspiration from rbenv which is how Ruby does this, so if you know Ruby then you'll understand it. When I understand better, I'll write another blog post


A while ago I noted that I should write about my shell functions for creating and managing Python Virtual Environments. Recently I was helping my boss set up some python virtual environments for some different projects, and I couldn't remember how to use virtualenv directly.

I really wanted to just share my wrappers with him, and I found that they weren't quite ready to share because I hadn't finished documenting them.

This post is to remedy that.

Read more…

Converting latin-1 To utf-8 with Python

Tonight I finally converted all the Glossary pages in my mirror of the Jargon File into Unicode (utf-8 encoding) so that they will transmit and display properly from GitHub Pages (or any other modern web server). It was a fairly trivial thing to do in the end, but I am likely to need to repeat this for other things at work, so I'm blogging it.

The Jargon File was converted into XML-Dockbook and Unicode for version 4.4.0, but ESR only converted the front- and back-matter, not the Glossary entries (i.e. the actual lexicon). Those are still latin-1 (ISO-8859-1). And although the HTML rendition begins with the correct header declaring this:

<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>

The pages are actually served from catb.org as Unicode (utf-8). For instance, compare /dev/null on catb.org with my mirror of /dev/null.

Read more…

Journald experiments - Testing systemd's logger: speed and buffering

I've been having good natured arguments at work about whether it's the End of the World that we are at last switching away from Scientific Linux 6 and it's System V style init scripts, to CentOS 7, which uses systemd.

My own opinion is that systemd is pretty cuspy. It's not perfect, but nor is it some great hulking monolithic monster come to destroy the Unix Way in the Linux world. It offers many worthwhile improvements and I've enjoyed using it in openSUSE for years now. I look forward to switching away from the hair-ball of wet SysV init scripts with clumsy precedent semantics and manual service recovery.

Now, I don't want to throw my hat into the ring on the pro's and con's of systemd having replaced the start-up infrastructure (and a lot of other systems besides) on Linux-based operating systems. Enough has been said already on that front, by many more experienced than I, and further argument is pointless: whichever camp you're in, you won't be convinced of the other sides point of view by now.

However there is one argument against systemd that I'm not so sure about: journald and it's past issues:

  • alleged buffering of logs, making diagnostics and debugging on time-critical services difficult or impossible
  • binary log files which can be corrupted, and then not useful thereafter (because they're binary)
  • volatile storage, so that your logs are gone when you want them the most: after an unplanned reboot

I'll be spending a few days experimenting with journald in these areas, to see if it's as bad now as it was five years ago when concerns like these were being raised.

In this post I want to look at the journald daemon / journalctl log viewer a bit, from the point of view of buffering output, whether and where it could be occurring, and what the implications might be as a web sysop.

This is a medium-long post, with about 23 minutes of terminal output recordings (in text, using asciinema) and is about 2⅓MB to download. It's also about half-an-hour's read on top of that.

Read more…

4-bit Rules of Computing, Part 4

Here is the fifth part of my blog series expanding on my 4-bit rules of computing.

In this post: rules 7, 8, and 9; which discuss testing and debugging. They are all related in a way: having to do with making good-quality craft work. Because, as much as computer people like to believe that we're "engineers" or that this is "computer science", we're not really. We're crafts people, in a profession that's still very young and finding its roots and methods in order to be consistently successful.

I'm definitely not trying to pretend I'm an "engineer". For real rigour, there is much more required than a few simple rules. But these are some realistic and humble rules in the area of testing that I aim to stick to.

Read more…

Learning DVCS Workflow - 0

If you take look at the revision history for some of my projects on GitHub, you'll see that I have a fairly messy track-record!

A few times I have successfully used Magit's ability to interactively stage/unstage Hunks or parts of Hunks, to make my commits a bit more clean and sensible, but not always.

The real problem is: I haven't been using Branches. It's because I haven't studied how to use them effectively, and in the past they've been scary.

But for the future, and especially for my dotfiles, I'd like to be able to read through the commits and make sense of them after. Also I'd like to be sure when I'm committing that I don't make any unplanned master changes and break things.

I also tend to work by myself on these projects, but I'll often go on a tangent, or start a blog post and then put off finishing it while another, different idea is developed and maybe even published first. Ideally I should be able to track these things separately.

So I need to learn: how to do revision control workflow with branches, properly.

Read more…

4-bit Rules of Computing, Part 3

Here is the forth part of my blog series expanding on my 4-bit rules of computing.

Previously in Milosophical Me: Mike was reflecting on Comments, both in source code and in social media, and had come to the conclusion that they are to be avoided, that they can be more harmful than helpful, and that source codes (and people) should be allowed to speak for them selves.

There is an exception to Rule 5 (Rule 0 allows for this): doc-comments. In this post I explore what they are, how they differ from regular comments, and how to use them to assist your fellow hackers.

Read more…