Sysadmin Appreciation Day 2024
Posted: 2024-07-26 10:50:42 by Alasdair Keyes
Once again, it's Sysadmin day.
Give thanks to your sysadmin. If you are a sysadmin, have a beer.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Installing LineageOS on ancient device
Posted: 2024-06-18 16:56:26 by Alasdair Keyes
This week I finally upgraded my phone from my ancient OnePlus 2 I bought in 2015 to a more modern Samsung.
For years, I've wanted to install LineageOS, an open Android variant. I always held off from installing it on my only phone in case I bricked it and was left up the creek without a paddle.
Now that I have the OnePlus 2 spare, I decided to finally go for it. Unfortunately, the device is now so old that LineageOS no longer hosts the installation images for it.
Thankfully, with https://archive.org/ I was able to gather all this data and thought it might be useful for others in the same situation. I will use my OnePlus 2 as an example, but obviously substitute this for your device.
-
Visit https://wiki.lineageos.org/devices/ - Using the
Filter Optionsunselect theHide discontinued devicescheck box then find your device. If it does not appear LineageOS never supported your device. -
Navigate to the page for your device (For me https://wiki.lineageos.org/devices/oneplus2/). From here it's worth noting the latest LineageOS version that was available for your phone under
Previously supported versions(17.1) and you will have access to the installation instructions via theInstallationlink. (The installation instructions will tell you to build the image from source, but ignore that, we will get the pre-built image). -
As the build images are no longer kept we will have to go to https://archive.org/ to hunt for it. When at archive.org enter the URL https://wiki.lineageos.org/devices/oneplus2/ into the Wayback machine search box. You will need to browse around all the snapshots to find the last snapshot that has the heading
Get LineageOS for the OnePlus 2and aGet the builds herelink.
(Using a manual binary search on the snapshots is probably the quickest way to find this).
For my device, this URL was https://web.archive.org/web/20210518101444/https://wiki.lineageos.org/devices/oneplus2/
-
From this archived page click on the
Get the builds herelink. This page doesn't link to the image files any more, but it does show the image names, which we can use. -
Go back to https://archive.org/ and in the search make sure
Search Metadatais selected and enter the image namelineage-17.1-20210605-nightly-oneplus2-signed
If archived, it will show the files available. If it doesn't you're probably out of luck. There are other sites that provide builds, but I would only trust archive.org to not have tampered with the images.
You may wish to play around with the search function and enter just nightly-oneplus2-signed and scan the results for an even later version that may have been archived. The numbers in the filename 20210605 are the concatenated year/month/day that the image was generated.
For the Last LineageOS 17.1 build on the OnePlus 2 this page is https://archive.org/details/lineage-17.1-20210605-nightly-oneplus2-signed_202108 - You can download the image via HTTPS but be a friend to archive.org and use the Torrent instead to help save them data costs.
Follow the installation instructions we found in Step 2.
The TWRP recovery image and Google Apps image are still available at LineageOS through the links provided in the installation instructions. From here you should have everything you need to install LineageOS on your ancient device.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
HMRC Basic PAYE Tools P60 generation error on Linux
Posted: 2024-04-12 09:52:59 by Alasdair Keyes
TL;DR if you are just looking for the fix scroll down to see the The Fix section.
As I run a UK company, I have to supply various bits of information to HMRC (the UK tax office) for payroll.
HMRC offer a free tool called Basic PAYE Tools for small businesses to make this possible (https://www.gov.uk/basic-paye-tools). It's a great tool, without it small companies would have to shell out money for proprietary solutions just to fulfil basic payroll/tax obligations. The tool is available for Windows, Mac and Linux. This is specifically for the Linux version, but a similar bug/fix may work for Mac.
The tool is a binary rti.linux that starts a web-server which listens on a http://127.0.0.1:46729/. A browser is then opened to automatically connect to that URL to provide the user with a GUI. The server stores data in a local sqlite database. When any actions are performed that require HMRC to be notified (such as paying employees), these transactions are stored locally and then sent via a batch process to HMRC.
As we have just passed into the 24-25 tax year, I have to 'close' the old year so that I can start the new one. Part of this process is to generate a P60 form for all employees. However, when doing this, the page just appeared to refresh and no form was displayed. This action is not one that requires any transfer of information to HMRC as the P60 is just a summary of an employees payments and tax contributions for that tax year.
A Google didn't show any similar issues and there was no easy 'File bug report' option. The only reporting is to call up HMRC on the phone. I didn't really want to try explaining the issue over the phone so I thought I'd investigate myself.
I could find no information about log files for the tool so I found the process of the rti.linux and looks in /proc/ for any possible logs.
$ ls -al /proc/24876/fd/
total 0
dr-x------ 2 user user 0 Apr 12 09:56 .
dr-xr-xr-x 9 user user 0 Apr 12 09:52 ..
lr-x------ 1 user user 64 Apr 12 09:56 0 -> /dev/null
l-wx------ 1 user user 64 Apr 12 09:56 1 -> /home/user/.xsession-errors
lrwx------ 1 user user 64 Apr 12 09:56 10 -> 'socket:[147745]'
lrwx------ 1 user user 64 Apr 12 09:56 11 -> 'socket:[148555]'
lrwx------ 1 user user 64 Apr 12 09:56 12 -> 'anon_inode:[eventfd]'
lrwx------ 1 user user 64 Apr 12 09:56 13 -> 'socket:[148556]'
lrwx------ 1 user user 64 Apr 12 09:56 14 -> 'socket:[147746]'
lrwx------ 1 user user 64 Apr 12 09:56 15 -> 'socket:[147747]'
lrwx------ 1 user user 64 Apr 12 09:56 18 -> 'socket:[147819]'
l-wx------ 1 user user 64 Apr 12 09:56 2 -> /home/user/.xsession-errors
lrwx------ 1 user user 64 Apr 12 09:56 21 -> 'anon_inode:[eventfd]'
lr-x------ 1 user user 64 Apr 12 09:56 3 -> /dev/urandom
l-wx------ 1 user user 64 Apr 12 09:56 4 -> /tmp/rti.log
lrwx------ 1 user user 64 Apr 12 09:56 5 -> 'anon_inode:[eventfd]'
lr-x------ 1 user user 64 Apr 12 09:56 6 -> 'pipe:[144859]'
l-wx------ 1 user user 64 Apr 12 09:56 7 -> 'pipe:[144859]'
lrwx------ 1 user user 64 Apr 12 09:56 8 -> 'socket:[144860]'
lrwx------ 1 user user 64 Apr 12 09:56 9 -> 'socket:[144024]'
The /tmp/rti.log shows one entry.
24876 Server has asked us to open_file
And /home/user/.xsession-errors shows something a little more helpful.
evince: error while loading shared libraries: libjpeg.so.62: failed to map segment from shared object
So it looks like an error in libjpeg.so.62, but nothing specific.
To get more information, I ran strace to find more about what was being called. (The read() call data is truncated to save space).
$ strace -Ff -v -s 1000 -p 24876
[pid 25497] openat(AT_FDCWD, "/home/user/HMRC/payetools-rti/libjpeg.so.62", O_RDONLY|O_CLOEXEC) = 3
[pid 25497] read(3, "\177ELF\2\1\----TRUNCATED----\0)\0\0\0", 832) = 832
[pid 25497] newfstatat(3, "", {st_dev=makedev(0xfd, 0x1), st_ino=23609456, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=848, st_size=432128, st_atime=1712911963 /* 2024-04-12T09:52:43.275396143+0100 */, st_atime_nsec=275396143, st_mtime=1602102343 /* 2020-10-07T21:25:43+0100 */, st_mtime_nsec=0, st_ctime=1712611938 /* 2024-04-08T22:32:18.204486045+0100 */, st_ctime_nsec=204486045}, AT_EMPTY_PATH) = 0
[pid 25497] mmap(NULL, 434200, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f318954d000
[pid 25497] mprotect(0x7f3189551000, 413696, PROT_NONE) = 0
[pid 25497] mmap(0x7f3189551000, 241664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = -1 EACCES (Permission denied)
[pid 25497] close(3) = 0
[pid 25497] writev(2, [{iov_base="evince", iov_len=6}, {iov_base=": ", iov_len=2}, {iov_base="error while loading shared libraries", iov_len=36}, {iov_base=": ", iov_len=2}, {iov_base="libjpeg.so.62", iov_len=13}, {iov_base=": ", iov_len=2}, {iov_base="failed to map segment from shared object", iov_len=40}, {iov_base="", iov_len=0}, {iov_base="", iov_len=0}, {iov_base="\n", iov_len=1}], 10) = 102
[pid 25497] exit_group(127) = ?
[pid 25497] +++ exited with 127 +++
Here we see /home/user/HMRC/payetools-rti/libjpeg.so.62 is opened for reading, the data is read and then mmap() and mprotect() are run, the file is closed and then we see the log error above is written to file handle #2 (/home/user/.xsession-errors) before the process exits with a non-zero code.
This indeed looks to be the issue, I don't know enough about the codebase to determine if it is corrupt or perhaps it wasn't updated when the rest of the code? It's very likely a FOSS library. So I checked if Debian supplied a version.
$ apt-file search libjpeg.so.62
libjpeg62-turbo: /usr/lib/x86_64-linux-gnu/libjpeg.so.62
libjpeg62-turbo: /usr/lib/x86_64-linux-gnu/libjpeg.so.62.3.0
Great, maybe I can use that?
The Fix
-
Make sure paye-tools is closed.
-
Install the
libjpeg62-turbopackage
It turns out I already had the libjpeg62-turbo package installed, but if you don't you can use.
apt install libjpeg62-turbo
I checked if the files were the same.... it turns out they're different versions.
$ sha256sum /home/user/HMRC/payetools-rti/libjpeg.so.62.paye-tools-version /usr/lib/x86_64-linux-gnu/libjpeg.so.62
4f3446bc4c2a2d3c75b7c62062305ff8c5fcdaa447d5a2461d5995d40f728d00 /home/user/HMRC/payetools-rti/libjpeg.so.62
dad87949ccad2be7e40a02986306087fdcfb35ccaadd59aea923a3f96d290eec /usr/lib/x86_64-linux-gnu/libjpeg.so.62
- Back-up and link to the system version
$ cd /home/user/HMRC/payetools-rti
$ mv libjpeg.so.62 libjpeg.so.62.orig
$ ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so.62 libjpeg.so.62
I started the tool back up and I was able to get the P60.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Simple Local LLM AI via docker-compose
Posted: 2024-03-13 16:21:36 by Alasdair Keyes
I've been playing about with local LLM (Large Language Model) AIs.
I knocked together this docker-compose.yml file to help people get started with Ollama with a nice Open-WebUI front-end to have the "joy" of AI, but locally.
It's available through a Gitlab snippet, or you can copy and paste from below. https://gitlab.com/-/snippets/3687211
---
# Created by Alasdair Keyes (https://www.akeyes.co.uk)
# * `docker-compose up`
# * Visit http://127.0.0.1:3000 to create account and login
# * Click 'Select a model'
# * Enter the model name to use. Click the link on the page to see all. `llama2` or `llama2-uncensored` are suitable first options.
# * Chat
version: '3'
services:
ollama:
image: "ollama/ollama"
volumes:
- ollama-data:/root/.ollama
# Uncomment ports to allow access to ollama API from the host
# ports:
# - "127.0.0.1:11434:11434"
open-webui:
image: "ghcr.io/open-webui/open-webui:main"
depends_on:
- ollama
ports:
- "127.0.0.1:3000:8080"
environment:
- "OLLAMA_BASE_URL=http://ollama:11434"
volumes:
- open-webui-data:/app/backend/data
volumes:
ollama-data:
open-webui-data:
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Best Albums of 2023
Posted: 2024-01-01 11:17:12 by Alasdair Keyes
My most enjoyed albums of 2023...
- Godsmack - Lighting Up The Sky
- Health - Rat Wars
- Neil Young and Crazy Horse - All Roads Lead Home
- Extreme - Six
- Filter - The Algorithm
Special mentions Jungle - Volcano, which just missed out a place in the top five. Additional mentions to Metallica - 72 Seasons, Aphex Twin - Blackbox Life Recorder 21f / In a Room7 F760 as it's been a while since each had a release but neither quite had the spark I was hoping for.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Updating Dell BIOS on a machine without Windows Installed
Posted: 2023-12-23 14:53:03 by Alasdair Keyes
I recently had a Dell laptop that required a BIOS update, but the BIOS installers were only for Windows and my machine was running Debian.
This posed a problem as to how to apply the update. I didn't want to go swapping out hard-drives to install Windows, or even wipe the Linux installation to install Windows and then re-install Linux after.
In the end I found that I could start a Windows 10 installation process, drop to a shell and run the BIOS update.
The steps are as follows...
- Download a Windows 10 ISO image from Microsoft. Write it to a DVD or use software such as Rufus (https://rufus.ie/) to create a Windows 10 Bootable USB drive.
- Download the BIOS update
.exefiles and put them onto a second USB stick. - Boot into the Windows 10 installer.
- At the first screen that asks you for languages, press
SHIFT+F10, a Windows Command prompt appears. - Insert the second USB stick with the update into a second USB port. Wait 5 seconds for the drive to be auto-mounted, Windows will assign it drive letter
E:. - In the command prompt type
e: <ENTER>. - Run the executable file that you downloaded. The update will run and then reboot into the BIOS update process as normal.
After the update, you can remove the USB sticks and reboot back into Linux.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Sysadmin Appreciation Day 2023
Posted: 2023-07-28 07:59:10 by Alasdair Keyes
It's that time of year again - Give thanks to your sysadmin.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Ansible/Jinja2 array concatenation changes in Debian Bookworm
Posted: 2023-07-24 21:50:19 by Alasdair Keyes
When I was first getting into Ansible, I read this article https://www.jeffgeerling.com/blog/2017/adding-strings-array-ansible by Jeff Geerling on how to add items to arrays a playbook using Jinja2 and I ended up incorporating it into my playbooks.
It's worth reading the article but essentially the code is...
hosts: localhost
connection: local
gather_facts: false
vars:
my_messages: []
tasks:
- name: Add to array
ansible.builtin.set_fact:
my_messages: "{{ my_messages }} + ['Message added to array: {{ item }}']"
loop:
- element one
- something else
- 3
- false
- name: Display messages
ansible.builtin.debug:
var: my_messages
This produced the following debug output
ok: [127.0.0.1] => {
"my_messages": [
"Message added to array: element one",
"Message added to array: something else",
"Message added to array: 3",
"Message added to array: False"
]
}
I found this useful for any post-run reminders. The debug block was added at the end of the playbook and could prompt me to perform any other tasks that might be required.
After I upgraded from Debian Bullseye (11) to Bookworm (12) this feature was now no-longer working correctly. The code wouldn't error but the output would be one continuous string including the [] characters.
ok: [127.0.0.1] => {
"my_messages": "[] + [ 'Message added to array: element one' ] + [ 'Message added to array: something else' ] + [ 'Message added to array: 3' ] + [ 'Message added to array: False' ]"
}
The change has occurred somewhere between Ansible 2.10.7 (with Jinja 2.11.3) and Ansible 2.14.3 (with Jinja 3.1.2).
To resolve this, the correct code is now...
hosts: localhost
connection: local
gather_facts: false
vars:
my_messages: []
tasks:
- name: Add to array
ansible.builtin.set_fact:
my_messages: "{{ my_messages + ['Message added to array: ' + item | string] }}"
loop:
- element one
- something else
- 3
- false
- name: Display messages
ansible.builtin.debug:
var: my_messages
The only line that has changed is the my_messages: fact setting.
The array manipulation is now performed entirely within the {{ }} Jinja2 tags. This has a couple of knock-on effects that you will need to be aware of...
-
You cannot use a loop variable with
{{ item }}method as you will be using tags within tags and Ansible will throw an undefined variable error. You will have to concatenate your string/variables using the+operator. -
Because you are not using the
{{ }}tags (because of point 1) but instead the concatenation operator+. It expects that you are only concatenating strings. As such you will need to ensure that any variables being joined are strings. If they are not, convert them with the| stringmodifier.
The second example shows both these methods being used to work correctly with later versions of Ansible/Jinja2
Depending on your use-case it's a little more messy, but won't take too much effort to convert.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Migrating calendars and contacts from Nextcloud to Radicale
Posted: 2023-04-09 08:53:16 by Alasdair Keyes
I've been running Nextcloud (https://nextcloud.com/) for several years (and Owncloud (https://owncloud.com/), prior to that) to provide my personal CalDAV calendar and CardDAV contact services.
My original intention was to make use of the other services that Nextcloud provides but as it turns out, I never have. It's a little overkill to run a full instance just for this functionality, it's also a big security footprint to keep updated and maintained when not making use if it to the full so I started to look for other solutions.
I tested out Radicale V3 and it seemed to do just what I needed. It provides both those services and is also included in the Debian repos, so it provides an easier install and update.
Although I tested out Radicale on Debian 11 which has V3, the server I will use for this is Debian 10 so I only get Radicale V2. Both seemed to work well, but this article is more about V2.
The configuration I chose was to run Radicale bound to localhost with htpasswd authentication and an Nginx reverse proxy in front to provide access via the internet. This seems to be the most basic and easiest setup. Although the bulk of this article is taken from the Radicale documentation, there are a few tweaks and changes included that I had to/wanted to make.
- Radicale V2 docs https://radicale.org/v2.html
- Radicale V3 docs https://radicale.org/v3.html
Note: Radicale refers to each calendar/contact entry as a collection, so you will see that terminology used here.
- Install
The python3 libraries are not requirements of the Radicale package, but if you're using htpasswd auth you will need them as they're required for it's htpasswd and bcrypt processing.
apt install radicale python3-passlib python3-bcrypt
- Configure Radicale
Edit /etc/radicale/config and ensure the following config exists.
[auth]
type = htpasswd
htpasswd_filename = /etc/radicale/users
htpasswd_encryption = bcrypt
delay = 1
- Create the user
# htpasswd -B -c /etc/radicale/users yourusername
New password: ************
Re-type new password: ************
# chmod 640 /etc/radicale/users
# chown radicale: /etc/radicale/users
- Set up Nginx
This is straight from Radicale docs. The docs also provide a copy/paste Apache config too.
location /radicale/ { # The trailing / is important!
proxy_pass http://localhost:5232/; # The / is important!
proxy_set_header X-Script-Name /radicale;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass_header Authorization;
}
If you're going to be importing existing data, you'll likely need to extend the reverse proxy timeouts as my calendar import took quite a few minutes. Your timeouts may vary but as an indicator my 1.2M calendar file took about 2-3 minutes to import.
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_send_timeout 300;
Restart everything
# systemctl restart radicale.service
# systemctl reload nginx.service
- Download calendars and contacts from Nextcloud.
Once logged into your Nextcloud instance, you can get a single .ics and .vcf file for your calendar/contacts by going to the URL with ?export added to the end.
- https://yournextcloudserver.com/remote.php/dav/addressbooks/users/username/Contacts/?export
- https://yournextcloudserver.com/remote.php/dav/calendars/username/personal/?export
It should go without saying, not to make any changes to your contacts/calendars between making the backups here and setting up your clients to use the new Radicale server.
- Creation/Uploading of calendars
Login to the web interface that's provided by the NGINX/Apache config you created above. e.g. https://myradicaleserver.com/radicale/
- Radicale V3 - You can upload your .ics and .vcf files through the web interface and be done with it
- Radicale V2 - You will need to create new collections and then overwrite with your backups.
The collections created use UUIDs, I didn't want to have to bother with those so I changed it to nicer paths...
# cd /var/lib/radicale/collections/collection-root/yourusername
# ls
# ls -al
total 188
drwxr-x--- 4 radicale radicale 4096 Apr 8 20:13 .
drwxr-x--- 4 radicale radicale 4096 Apr 8 19:43 ..
drwxr-x--- 3 radicale radicale 167936 Apr 8 20:40 55f41fed-eccb-4feb-a460-69b1745d2c02
drwxr-x--- 3 radicale radicale 12288 Apr 8 20:16 6a2b1bfd-ca46-45e4-8dcf-861b448c519f
# mv 6a2b1bfd-ca46-45e4-8dcf-861b448c519f contacts
# mv 55f41fed-eccb-4feb-a460-69b1745d2c02 calendar
# ls -al
total 188
drwxr-x--- 4 radicale radicale 4096 Apr 8 20:13 .
drwxr-x--- 4 radicale radicale 4096 Apr 8 19:43 ..
drwxr-x--- 3 radicale radicale 167936 Apr 8 20:40 calendar
drwxr-x--- 3 radicale radicale 12288 Apr 8 20:16 contacts
Refreshing the web UI will show the updated paths.
- Import the dumps to Radicale
# curl -u 'yourusername:plaintextpassword' -X PUT https://yourradicaleserver.com/radicale/username/calendar --data-binary @personal-2023-04-08.ics
# curl -u 'yourusername:plaintextpassword' -X PUT https://yourradicaleserver.com/radicale/username/contacts --data-binary @Contacts-2023-04-08.vcf
Reresh the Web UI and you will notice that the name/description/colours you set will have been changed or removed. Use the Edit link to set them again.
That's it, you then just need to configure your clients to use the new calendars/contacts. I use Thunderbird for my PC and DavX5 on my phone and both worked with no issues.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Disabling syslog debugging on open-fprintd
Posted: 2023-04-02 19:23:47 by Alasdair Keyes
I recently installed python-validity (https://github.com/uunicorn/python-validity) and open-fprintd (https://github.com/uunicorn/open-fprintd) to get the fingerprint reader working on a Thinkpad T480s. (The standard fprintd package doesn't support the fingerprint reader device on the T480s).
After setting it up I noticed a lot of debug information in syslog (example truncated)...
Apr 2 09:11:00 hostname01 open-fprintd: >tls> 17: 4b00000b0053746757696e64736f7200
Apr 2 09:11:00 hostname01 open-fprintd: >cmd> 1703030050c00a...a82185cc9399d30625ee3c1451f
Apr 2 09:11:00 hostname01 open-fprintd: <cmd< 1703030050b7a4a...cdb0d7f97fa67b6337329
Apr 2 09:11:00 hostname01 open-fprintd: <tls< 17: 0000030002000b00000...6757696e64736f7200
To disable debug without changing the installed systemd files, run the following to create a custom open-fprintd.service file to override it.
sed 's/\(ExecStart.*\)\-\-debug\(.*\)/\1 \2/' /lib/systemd/system/open-fprintd.service > /etc/systemd/system/open-fprintd.service
After a reboot it'll stop spewing that data out.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
© Alasdair Keyes
IT Consultancy Services
I'm now available for IT consultancy and software development services - Cloudee LTD.
Happy user of Digital Ocean (Affiliate link)
Version:master-53c82addfa
