Author: mark

time tracking by writing a rofi plugin / learning C so I can finally send accurate invoices

I’m a freelance software developer. As a lifelong scatterbrain, the part of my job I hate the most is time tracking. Maybe this particular personal failing ought to have precluded me from becoming a freelancer but everything seems to be working out okay.

I bill clients by the hour, and I don’t want to either (a) rip the piss or (b) shortchange myself. But when it comes time to send invoices and I realise I have been delinquent in recording my hours, of course I err on the side of the latter.

The accounting software I use (freeagent) has a handy web / app based time entry system complete with timers, but I’ve always found having to log in and mess around with a web form a bit … listen, I don’t want to make excuses. It’s not you, it’s me.

Anyway, the time has come for me to sort myself out. Enter my super niche January side project: fatt, a rofi-based freeagent timer control!

The gist of the video above, if you can’t or won’t watch it, is that I can now create, start and stop FreeAgent timers in a custom rofi mode.

Interesting bits

Learning C

As a total newbie to C (save for some fannying about with the odd microcontroller), writing a plugin for rofi was a pretty straightforward and practical way to jump in. qball’s blog post was helpful and enlightening, as was examining the code of a few other plugins (calc, file-browser-extended, blocks).

That said, I had given myself a fairly firm deadline on this (24th Jan) so I cheesed it a wee bit by deciding that the rofi plugin would just be a frontend for a CLI app that would do the heavy OAuth2 / network lifting.

The documentation for glib is extensive and it will probably be my starting point for any future non-microcontroller C project. Since I was writing a frontend for a CLI, I determined quickly that I wanted to fork, and I would need g_spawn_async_with_pipes. The best example code I found was in Gnome itself, specifically for using passwd as a CLI backend for the gnome-control-center UI.

I stripped that back into something general purpose, that is used like this:

Then, in my callback functions, I read the CLI output line by line into an array of “rows” to be displayed by rofi. I could really define that however I like – the rofi plugin ABI just asks me to define some functions along the lines of (pseudocode) get_num_entries and get_display_value(entryId) – so I opted to define a Row struct which also stores the freeagent object ID relating to each row.

OAuth2 client on the CLI

FreeAgent’s API can be accessed through OAuth2 only, I can’t just use account-specific secret tokens to access my own account. There is probably a reasonable excuse for this, although I note FreeAgent itself appears to be hosted on AWS – presumably there’s a dev somewhere in Edinburgh with a file in the format aws_access_key_id:aws_secret_access_key in their /etc folder that would allow them to do what they want with my data.

So anyway, I have to register as a developer, register an app and use localhost as the callback URL for that app. When my cli app goes to get bearer and refresh tokens, it has to launch an HTTP server which will parse the tokens from GET parameters. I do it like this:

Displaying my daily “money earned” in i3bar

Well, why not? The CLI command takes a list of the current day’s timeslips and multiplies them by the hourly rate of their related task. i3status is then piped through a script which prepends this figure to my other vital stats – wifi speed, load average, etc.

I have some ideas for this that are as yet unrealised – for example, on the easy side, it could turn green when a timer is on. Or, on the very hard side, something could detect I’ve had twitter open for 5 minutes and automatically stop a timer, turning my status bar red and flashing or something. Don’t know where to start with that one, hints in the comments please.

Some people have informed me that this is not only depressingly capitalistic, but also psychologically inferior to a progress bar. That is why I also submitted a pull request to i3status to make their example perl script chainable – so I could return a vague promise to also show a burndown sparkline for any serious side project or savings goal that I start working on. I also reduced the refresh rate to once a minute as it definitely felt a bit psychopathic to be getting a visual update every time I made seven pence.

Postponing learning Golang, for now

I’ve already used the deadline I set as an excuse for writing a CLI at all, but I actually originally intended to write the CLI using Golang – yet another language I’ve intended to learn properly for a long time. But the barrier of trying to set up the whole temporary HTTP server for OAuth2 in a language I’m not familiar with was frustrating enough that I decided that I would be better doing it quickly in PHP, since it would be very fast and let me get to the fun stuff (the C bit).

How do I feel after a week?

A week may be too soon to tell, but the rapid ability to toggle timers, combined with the cash counter in the status bar, gave me a real sense of either being “on” or “off”. That discouraged procrastination, which in itself was a good outcome that encouraged me to be diligent with the timer controls. As a result I had a pretty good, productive week and also all the boring timesheet admin did not need rushed in at 5pm on Friday (or 10am the following Monday). So I am pretty pleased with it.

Closing thoughts

In general I would always encourage anyone to spend time writing software that will only benefit themselves. At least for me, this sort of personal tool creation is the genuinely interesting thing about computer programming, way more than writing code in the service of our bosses’ attempts to generate economic value.

More specifically I can really recommend a rofi plugin as a really handy way to get input / listing / filtering taken care of for a productivity utility. If you see rofi as some kind of a super limited GUI framework it takes on quite an interesting shape, with lots of creative possibilities.

Dell XPS 2020 – How to get audio working on Linux

update 24/4: Apparently everything is broken on ubuntu 21.04 because of kernel options. someone helpfully posted instructions on recompiling your kernel here: https://blog.fts.scot/2020/07/04/dell-xps-2020-how-to-get-audio-working-on-linux/#comment-356 – it should actually be possible to make a dkms module for this that can be installed in the same way as below, if anyone feels up for it i’ll post a link.

update 16/12: Linux 5.10 has these drivers available out the box, the arch and manjaro repos now have the required config options enabled by default so the easiest thing to do would be up to date on your packages. minimum versions listed below

Recently took delivery of a Dell XPS 17 9700 to replace my “workhorse laptop”, of course intending to use Linux as the only operating system. Before doing anything at all I reformatted the disk and installed Manjaro before realising that the audio drivers are not yet in the mainline Linux kernel nor is the audio device really supported by any of the stable releases from the wider audio ecosystem (specifically alsa or pulse).

Long story short, before starting, you should understand that you’ll be running the very latest kernel, compiling a module for it and setting your package manager to use unstable repos, and still having to copy in files anyway, and there are risks associated with that. So if you feel like you can wait until Pulseaudio 14 and Linux 5.10 come out – I would recommend that!

(update 5/7: If you’re wanting to stay on Ubuntu 20.04 I have been informed by someone at Canonical that you can apt install linux-oem-20.04 and the drivers should work (i.e. you’ll see lots of devices in alsamixer) without doing the DKMS steps, which is good, but you will still require to get the latest alsa, pulseaudio and ucm yourself to make it all work.)

I have been able to get this working on Manjaro with Linux 5.7, 5.8 and 5.9 with the following other software:

  • pulseaudio-alsa 1.2.2
  • pulseaudio 13.99
  • alsa 1.2.3 including alsa-ucm-conf 1.2.3
  • dkms
  • sof-firmware 1.5.1 or 1.6

To build the driver on 5.7:

git clone https://github.com/maaarghk/soundwire-dkms
sudo mv soundwire-dkms /usr/src/soundwire-1.3.0
sudo dkms add soundwire/1.3.0
sudo dkms build soundwire/1.3.0
sudo dkms install soundwire/1.3.0

This is based on this Ubuntu package, which has patches with the Dell XPS hardware IDs added on top of the driver from here. It previously targeted 5.0 so I applied some patches for higher kernel versions than that.

To build the driver on 5.8 or 5.9:

git clone https://github.com/maaarghk/soundwire-dkms
git checkout latest-sofproject
sudo mv soundwire-dkms /usr/src/soundwire-1.4.0
sudo dkms add soundwire/1.4.0
sudo dkms build soundwire/1.4.0
sudo dkms install soundwire/1.4.0

This driver comes directly from Intel’s thesofproject repo with the only patches being related to dkms and old kernel configs.

Once the driver is built it is worth rebooting and checking alsamixer to see if you have more than 6 outputs – maybe need to press f6 to switch to sof-soundwire – there should be faders for Headphones / Speaker and not just HDMI ports. If you only have HDMI ports something has gone wrong, dmesg | grep 'snd\|soc\|sof'.

If you have an error about symbol version disagreements, probably your kernel has an old sof driver which should be disabled by creating /etc/modprobe.d/no-builtin-soundwire.conf with the contents blacklist snd_soc_skl.

Anyway, that’s not the whole story, alsa also needs ucm files to know how to interact with the driver. Those are not even merged into unstable alsa-ucm-conf yet so the following changes are liable to break / require repeating at some stage. Price you pay for living on the edge, it’s just like the olden days of Linux.

git clone https://github.com/thesofproject/alsa-ucm-conf
sudo cp -r alsa-ucm-conf/ucm2/sof-soundwire /usr/share/alsa/ucm2
sudo cp -r alsa-ucm-conf/ucm2/codecs/rt5682 alsa-ucm-conf/ucm2/codecs/rt700 alsa-ucm-conf/ucm2/codecs/rt711 alsa-ucm-conf/ucm2/codecs/rt715 /usr/share/alsa/ucm2/codecs 

alsaucm open sof-soundwire
alsaucm reload

Here’s hoping the exasperated sounding lead dev on this repo successfully completes a code review without popping a vein and these changes make it upstream sooner rather than later!

Using pulseaudio-git you may find that pulseaudio does not start up automatically, the solution to that is simple:

systemctl --user enable pulseaudio

Cheers to reddit user /u/yoyoyomama1 who gave me many clues by answering a barrage of messages asking for conf files off his system. I would have been unlikely to work this out without that help.

p.s. Whilst I’m here, brief review of the laptop – very good. Feels really solid, haven’t had any of the QC issues mentioned by anyone else. The 4k screen is really nice, and I have been very impressed by the battery life – for example I started working on this about 6 hours ago and it has involved several code compilations and many restarts, and my battery life is still above 30%. It helps that the screen is so bright that I can run it at 10% brightness and still feel like it is burning my eyes.

Possible method for retrieving backups from a Nexus 5X with the boot-loop bug

Hello fellow owners of fucked Nexus 5X devices,

I started getting boot-loops this morning and a cursory search on the interwebs tells me I’m far from the only one. From the unending of stories of abysmal support from LG I’m not optimistic that I’ll have a Nexus 5X for much longer and I was beginning to reckon I’d lost all data since my last backup (which was way too long ago). Fortunately I seem to be able to (reproducibly) get fifteen minutes out of it by (i) plugging it into a laptop; (ii) letting it loop for a while; and (iii) wedging it somewhere where it will get very warm very fast – specifically I put it between two cushions. Eventually it boots happily.

Maybe I’m just lucky and this won’t work for most people, but I figure that for those who are currently stuck with a bricked phone, this is worth trying as a last-ditch effort for recovering any information at all.

What this did not do is permanently fix my phone. If you’re thinking of giving this a go, bear in mind that it does only last fifteen minutes and there’s no guarantee this will continue to work more than three or four times (or even once). Also, performance is notably terrible (presumably due to thermal sensors throttling the CPU) so you have to plan ahead here and be patient so you don’t overload the CPU if you do get it booted.

However, with planning I was able to get just enough time to download some important backups onto my laptop. I’ll describe the steps required below.

Prerequisites

Install Android platform tools on your PC (I used a Mac on either Windows or Linux). You’ll be using adb to pull down the files you need.

If you’re not familiar with ADB, read some guides on the installation and basic concepts. Follow a tutorial to get it installed, there are loads.

Once you have it installed, run adb wait-for-device in the Terminal.

You’ll need a USB to USB-C cable if your PC doesn’t have a USB-C port.

Determine in advance what to back up

I had a few apps in mind which I knew would not be backed up on the internal storage but whose settings or status I wanted a copy of for post-factory-reset or replacement.

There is an adb backup command which lets you accomplish this. Before you run it it you need to know the ID of the app you want to backup. Visit Google Play Store and click “My Apps”; find the apps you want to backup and click on them. The id is shown in the URL bar.

Example: for Terminal Emulator, the url is https://play.google.com/store/apps/details?id=jackpal.androidterm  – meaning the id is jackpal.androidterm.

The command you want to run follows the format adb backup -f <appid>.ab <appid>; make a note of the commands you’ll need:

adb backup -f jackpal.androidterm.ab jackpal.androidterm
adb backup -f com.tpcstld.twozerogame.ab com.tpcstld.twozerogame

There’s another command, adb pull, which lets you download files from the internal storage. In my case I wanted to backup my photos (just incase there was anything not captured in Google Photos) and WhatsApp. The relevant folders are /sdcard/DCIM; /sdcard/Pictures and /sdcard/WhatsApp. Make a note of the commands you’ll need:

adb pull /sdcard/DCIM
adb pull /sdcard/Pictures
adb pull /sdcard/WhatsApp

Try these if you need them and maybe some helpful commenters will leave additional ideas. You also may be able to use Google to determine where an app you use stores its files.

Enable developer mode / USB debugging and run the commands

As soon as your phone boots, if you didn’t already do so before you started experiencing issues, you will need to enable USB debugging on your device. Memorise it:

  1. tap ‘Settings’ > ‘About’ > ‘Software Information’ then tap on ‘Build Number’ 7 times.
  2. Go back to ‘Settings’ then select ‘Developer Options’
  3. Tap and enable USB Debugging.

If you have done as you were told above and started adb wait-for-device you should pretty much immediately get a prompt on your screen asking you to allow debugging. Tick the box and click Accept.

As soon as the terminal in which you entered the adb command returns to the prompt, start entering in the commands you saved above. This will start pulling down files and backups to your computer ready to be restored some other time.

It’s also worth taking a look in the app list to see if there’s anything you missed. I had a Bitcoin wallet app installed – although I already had a backup I thought I might as well take the opportunity to grab another one just incase.

Report back

Let me know in the comments if this worked or did not work for you, or if you have any advice for anyone who tries this. Listing common app IDs or /sdcard data folders would be really useful.

Homemade Chinese takeaway-style curry

At the behest of my colleague Ross, I’m going to put a couple of recipes up. Here’s a homemade Chinese takeaway curry sauce that tastes pretty much like the real thing. If you are going to take a photo for your Instagram try to do a better job than this:

hello

Ingredients

1 tablespoon coconut oil (but vegetable oil will do).
1 onion.
4 cloves of garlic.
1 inch of ginger.
1 tablespoon of Chinese five-spice powder.
Half tablespoon of hot chilli powder.
1 tablespoon of light soy sauce.
400ml of chicken stock.
Half teaspoon of MSG.
2 tablespoons of flour.

Optionally:
Double all of the above and add 2 chicken breasts.

Method

Cut the onion into large chunks. Finely chop or grate the garlic and ginger. Fry the onion, garlic and ginger on a medium heat for three minutes or so, and then add the five-spice and chilli powder. If using the chicken, add now. Coat the contents of the pan with the powder and let it cook for up to a minute before adding the chicken stock and soy sauce. Don’t worry too much about the chicken being browned before adding the liquid – it’ll poach.

Let the stock come up to a boil before reducing to a simmer. Add in as much flour as you need to thicken it up – one tablespoon is usually enough. A whisk will help you get rid of any lumps. A tiny amount of MSG goes a long way – a half teaspoon is enough for this, even with chicken. Simmer for twenty minutes. If you leave it on for too long it and it gets slightly too thick just stir through a little bit of water.

Serve with egg fried rice and beer.