Simple Local LLM AI via docker-compose

Posted: 2024-03-13 16:21:36 by Alasdair Keyes

Direct Link | RSS feed


I've been playing about with local LLM (Large Language Model) AIs.

I knocked together this docker-compose.yml file to help people get started with Ollama with a nice Open-WebUI front-end to have the "joy" of AI, but locally.

It's available through a Gitlab snippet, or you can copy and paste from below. https://gitlab.com/-/snippets/3687211

---
# Created by Alasdair Keyes (https://www.akeyes.co.uk)
# * `docker-compose up`
# * Visit http://127.0.0.1:3000 to create account and login
# * Click 'Select a model'
# * Enter the model name to use. Click the link on the page to see all. `llama2` or `llama2-uncensored` are suitable first options.
# * Chat
version: '3'
services:

  ollama:
    image: "ollama/ollama"
    volumes:
      - ollama-data:/root/.ollama
# Uncomment ports to allow access to ollama API from the host
#    ports:
#      - "127.0.0.1:11434:11434"

  open-webui:
    image: "ghcr.io/open-webui/open-webui:main"
    depends_on:
      - ollama
    ports:
      - "127.0.0.1:3000:8080"
    environment:
      - "OLLAMA_BASE_URL=http://ollama:11434"
    volumes:
      - open-webui-data:/app/backend/data

volumes:
  ollama-data:
  open-webui-data:


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Best Albums of 2023

Posted: 2024-01-01 11:17:12 by Alasdair Keyes

Direct Link | RSS feed


My most enjoyed albums of 2023...

Special mentions Jungle - Volcano, which just missed out a place in the top five. Additional mentions to Metallica - 72 Seasons, Aphex Twin - Blackbox Life Recorder 21f / In a Room7 F760 as it's been a while since each had a release but neither quite had the spark I was hoping for.


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Updating Dell BIOS on a machine without Windows Installed

Posted: 2023-12-23 14:53:03 by Alasdair Keyes

Direct Link | RSS feed


I recently had a Dell laptop that required a BIOS update, but the BIOS installers were only for Windows and my machine was running Debian.

This posed a problem as to how to apply the update. I didn't want to go swapping out hard-drives to install Windows, or even wipe the Linux installation to install Windows and then re-install Linux after.

In the end I found that I could start a Windows 10 installation process, drop to a shell and run the BIOS update.

The steps are as follows...

  1. Download a Windows 10 ISO image from Microsoft. Write it to a DVD or use software such as Rufus (https://rufus.ie/) to create a Windows 10 Bootable USB drive.
  2. Download the BIOS update .exe files and put them onto a second USB stick.
  3. Boot into the Windows 10 installer.
  4. At the first screen that asks you for languages, press SHIFT+F10, a Windows Command prompt appears.
  5. Insert the second USB stick with the update into a second USB port. Wait 5 seconds for the drive to be auto-mounted, Windows will assign it drive letter E:.
  6. In the command prompt type e: <ENTER>.
  7. Run the executable file that you downloaded. The update will run and then reboot into the BIOS update process as normal.

After the update, you can remove the USB sticks and reboot back into Linux.


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Sysadmin Appreciation Day 2023

Posted: 2023-07-28 07:59:10 by Alasdair Keyes

Direct Link | RSS feed


It's that time of year again - Give thanks to your sysadmin.

https://sysadminday.com/


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Ansible/Jinja2 array concatenation changes in Debian Bookworm

Posted: 2023-07-24 21:50:19 by Alasdair Keyes

Direct Link | RSS feed


When I was first getting into Ansible, I read this article https://www.jeffgeerling.com/blog/2017/adding-strings-array-ansible by Jeff Geerling on how to add items to arrays a playbook using Jinja2 and I ended up incorporating it into my playbooks.

It's worth reading the article but essentially the code is...

hosts: localhost
connection: local
gather_facts: false

vars:
  my_messages: []

tasks:
  - name: Add to array
    ansible.builtin.set_fact:
      my_messages: "{{ my_messages }} + ['Message added to array: {{ item }}']"
    loop:
      - element one
      - something else
      - 3
      - false

  - name: Display messages
    ansible.builtin.debug:
      var: my_messages

This produced the following debug output

ok: [127.0.0.1] => {
    "my_messages": [
        "Message added to array: element one",
        "Message added to array: something else",
        "Message added to array: 3",
        "Message added to array: False"
    ]
}

I found this useful for any post-run reminders. The debug block was added at the end of the playbook and could prompt me to perform any other tasks that might be required.

After I upgraded from Debian Bullseye (11) to Bookworm (12) this feature was now no-longer working correctly. The code wouldn't error but the output would be one continuous string including the [] characters.

ok: [127.0.0.1] => {
    "my_messages": "[] + [ 'Message added to array: element one' ] + [ 'Message added to array: something else' ] + [ 'Message added to array: 3' ] + [ 'Message added to array: False' ]"
}

The change has occurred somewhere between Ansible 2.10.7 (with Jinja 2.11.3) and Ansible 2.14.3 (with Jinja 3.1.2).

To resolve this, the correct code is now...

hosts: localhost
connection: local
gather_facts: false

vars:
  my_messages: []

tasks:
  - name: Add to array
    ansible.builtin.set_fact:
      my_messages: "{{ my_messages + ['Message added to array: ' + item | string] }}"
    loop:
      - element one
      - something else
      - 3
      - false

  - name: Display messages
    ansible.builtin.debug:
      var: my_messages

The only line that has changed is the my_messages: fact setting. The array manipulation is now performed entirely within the {{ }} Jinja2 tags. This has a couple of knock-on effects that you will need to be aware of...

  1. You cannot use a loop variable with {{ item }} method as you will be using tags within tags and Ansible will throw an undefined variable error. You will have to concatenate your string/variables using the + operator.

  2. Because you are not using the {{ }} tags (because of point 1) but instead the concatenation operator +. It expects that you are only concatenating strings. As such you will need to ensure that any variables being joined are strings. If they are not, convert them with the | string modifier.

The second example shows both these methods being used to work correctly with later versions of Ansible/Jinja2

Depending on your use-case it's a little more messy, but won't take too much effort to convert.


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Migrating calendars and contacts from Nextcloud to Radicale

Posted: 2023-04-09 08:53:16 by Alasdair Keyes

Direct Link | RSS feed


I've been running Nextcloud (https://nextcloud.com/) for several years (and Owncloud (https://owncloud.com/), prior to that) to provide my personal CalDAV calendar and CardDAV contact services.

My original intention was to make use of the other services that Nextcloud provides but as it turns out, I never have. It's a little overkill to run a full instance just for this functionality, it's also a big security footprint to keep updated and maintained when not making use if it to the full so I started to look for other solutions.

I tested out Radicale V3 and it seemed to do just what I needed. It provides both those services and is also included in the Debian repos, so it provides an easier install and update.

Although I tested out Radicale on Debian 11 which has V3, the server I will use for this is Debian 10 so I only get Radicale V2. Both seemed to work well, but this article is more about V2.

The configuration I chose was to run Radicale bound to localhost with htpasswd authentication and an Nginx reverse proxy in front to provide access via the internet. This seems to be the most basic and easiest setup. Although the bulk of this article is taken from the Radicale documentation, there are a few tweaks and changes included that I had to/wanted to make.

Note: Radicale refers to each calendar/contact entry as a collection, so you will see that terminology used here.

  1. Install

The python3 libraries are not requirements of the Radicale package, but if you're using htpasswd auth you will need them as they're required for it's htpasswd and bcrypt processing.

apt install radicale python3-passlib python3-bcrypt
  1. Configure Radicale

Edit /etc/radicale/config and ensure the following config exists.

[auth]
type = htpasswd
htpasswd_filename = /etc/radicale/users
htpasswd_encryption = bcrypt
delay = 1
  1. Create the user
# htpasswd -B -c /etc/radicale/users yourusername
New password: ************
Re-type new password: ************
# chmod 640 /etc/radicale/users
# chown radicale: /etc/radicale/users
  1. Set up Nginx

This is straight from Radicale docs. The docs also provide a copy/paste Apache config too.

location /radicale/ { # The trailing / is important!
    proxy_pass        http://localhost:5232/; # The / is important!
    proxy_set_header  X-Script-Name /radicale;
    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header  Host $http_host;
    proxy_pass_header Authorization;
}

If you're going to be importing existing data, you'll likely need to extend the reverse proxy timeouts as my calendar import took quite a few minutes. Your timeouts may vary but as an indicator my 1.2M calendar file took about 2-3 minutes to import.

proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_send_timeout 300;

Restart everything

# systemctl restart radicale.service
# systemctl reload nginx.service
  1. Download calendars and contacts from Nextcloud.

Once logged into your Nextcloud instance, you can get a single .ics and .vcf file for your calendar/contacts by going to the URL with ?export added to the end.

It should go without saying, not to make any changes to your contacts/calendars between making the backups here and setting up your clients to use the new Radicale server.

  1. Creation/Uploading of calendars

Login to the web interface that's provided by the NGINX/Apache config you created above. e.g. https://myradicaleserver.com/radicale/

The collections created use UUIDs, I didn't want to have to bother with those so I changed it to nicer paths...

# cd /var/lib/radicale/collections/collection-root/yourusername
# ls 
# ls -al
total 188
drwxr-x--- 4 radicale radicale   4096 Apr  8 20:13 .
drwxr-x--- 4 radicale radicale   4096 Apr  8 19:43 ..
drwxr-x--- 3 radicale radicale 167936 Apr  8 20:40 55f41fed-eccb-4feb-a460-69b1745d2c02
drwxr-x--- 3 radicale radicale  12288 Apr  8 20:16 6a2b1bfd-ca46-45e4-8dcf-861b448c519f
# mv 6a2b1bfd-ca46-45e4-8dcf-861b448c519f contacts
# mv 55f41fed-eccb-4feb-a460-69b1745d2c02 calendar
# ls -al
total 188
drwxr-x--- 4 radicale radicale   4096 Apr  8 20:13 .
drwxr-x--- 4 radicale radicale   4096 Apr  8 19:43 ..
drwxr-x--- 3 radicale radicale 167936 Apr  8 20:40 calendar
drwxr-x--- 3 radicale radicale  12288 Apr  8 20:16 contacts

Refreshing the web UI will show the updated paths.

  1. Import the dumps to Radicale
# curl -u 'yourusername:plaintextpassword' -X PUT https://yourradicaleserver.com/radicale/username/calendar --data-binary @personal-2023-04-08.ics
# curl -u 'yourusername:plaintextpassword' -X PUT https://yourradicaleserver.com/radicale/username/contacts --data-binary @Contacts-2023-04-08.vcf

Reresh the Web UI and you will notice that the name/description/colours you set will have been changed or removed. Use the Edit link to set them again.

That's it, you then just need to configure your clients to use the new calendars/contacts. I use Thunderbird for my PC and DavX5 on my phone and both worked with no issues.


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Disabling syslog debugging on open-fprintd

Posted: 2023-04-02 19:23:47 by Alasdair Keyes

Direct Link | RSS feed


I recently installed python-validity (https://github.com/uunicorn/python-validity) and open-fprintd (https://github.com/uunicorn/open-fprintd) to get the fingerprint reader working on a Thinkpad T480s. (The standard fprintd package doesn't support the fingerprint reader device on the T480s).

After setting it up I noticed a lot of debug information in syslog (example truncated)...

Apr  2 09:11:00 hostname01 open-fprintd: >tls> 17: 4b00000b0053746757696e64736f7200
Apr  2 09:11:00 hostname01 open-fprintd: >cmd> 1703030050c00a...a82185cc9399d30625ee3c1451f
Apr  2 09:11:00 hostname01 open-fprintd: <cmd< 1703030050b7a4a...cdb0d7f97fa67b6337329
Apr  2 09:11:00 hostname01 open-fprintd: <tls< 17: 0000030002000b00000...6757696e64736f7200

To disable debug without changing the installed systemd files, run the following to create a custom open-fprintd.service file to override it.

sed 's/\(ExecStart.*\)\-\-debug\(.*\)/\1 \2/' /lib/systemd/system/open-fprintd.service > /etc/systemd/system/open-fprintd.service

After a reboot it'll stop spewing that data out.


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Group code reviews

Posted: 2023-01-29 16:03:38 by Alasdair Keyes

Direct Link | RSS feed


Whist working with a customer of mine, I was introduced to a way of code review that I hadn't seen before. I'm not sure if it's particularly unique to them, but in my experience it's fairly rare... group code review.

In all previous companies I've worked at/with, a pull request (PR) is created and has some key devs added for Code Review. For a speedy review a link to the PR was often thrown into the Dev Slack channel with a little prodding or begging to get the task done.

Now, most people in dev teams are busy; or at least if they're not too busy, they find almost anything more interesting than reviewing a PR. It can be tedious and if it's touching code that you're not familiar with, you can often end up spending some time trying to work out both what the code was doing and what the change is supposed to be doing. In the end it was often team leaders who ended up picking up the slack and performing the reviews to get code out the door. This puts an excessive burden on them when it could be shared more equally.

The process

This particular customer had group code reviews once in the morning after stand-up and then an optional code review mid afternoon if required. My first thought was that this would cause undue disruption to work flow but over time I saw immense benefit in this approach and very little drawback.

The developer that wrote the code would create a PR as normal. At the review, they would share their screen on a famous git-hosting provider. Usually they would give a sentence about the ticket and what they were trying to achieve. Due to stand-ups and backlog planning, everyone usually had a fairly good idea on this already. The dev would proceed to go through the changes on their branch, giving a run down of the code as they went. Questions or points could be made by anyone and a discussion could be had amongst the group. If new tests were made, they would also be run through briefly so that other devs could see what was being tested, whether it was suitable/adequate and any edge cases that might fall through the cracks and need to be added. (It also required that the tests passed!)

Any changes that were required and agreed upon by the team were added to the PR and the dev would go back and implement the changes.

If the changes needed were minor, they were often approved by a senior dev and scheduled for release. Larger changes would have to have a new PR created and go back into the code review process. The coder would only have to go through the changes from the last code review, not the whole shebang.

The benefits...

Things you think are bad... but they're not as bad as you think...

So that's my review on group code-review. If you're looking for possible improvements to your process I strongly advise you give it a shot. Do it for a month or so and see how you get on. You can always tweak the process based on your needs and fall back to your previous ways if you don't like it.


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

Adding custom Firefox config with Ansible

Posted: 2023-01-26 21:05:42 by Alasdair Keyes

Direct Link | RSS feed


I've been writing Ansible playbooks to build my laptop and I came across a problem with applying my custom Firefox user.js config file to a new build.

Firefox won't create config/profile directory until it's started for the first time. During this first start a random profile directory is created in which the user.js file needs to be placed e.g /.mozilla/firefox/wxyz1234.default-release/.

As such you can't just install user.js with a basic file copy.

To counter this I wrote the following play so that if there was no Firefox profile folder, Ansible will start Firefox, allow it to create the profile folder and then kill it. It will then search for the newly created profile folder and install the file.

# Install Firefox
- name: Install Firefox
  become: true
  package:
    name:
      - firefox
    state: present

# Check if a profile folder exists
- name: Check Firefox config folder
  find:
    paths: "{{ ansible_user_dir }}/.mozilla/firefox"
    patterns: '^.*\.default-release'
    use_regex: true
    file_type: directory
  register: firefox_config_folder_search

# If profile folder doesn't exist, start Firefox
- name: Starting Firefox
  shell:
    cmd: "firefox &"
  changed_when: false
  when: firefox_config_folder_search.matched == 0

- name: Waiting for Firefox to start
  command: sleep 10s
  changed_when: false
  when: firefox_config_folder_search.matched == 0

# Kill Firefox
- name: Killing Firefox
  shell:
    cmd: kill $(pidof firefox)
  changed_when: false
  when: firefox_config_folder_search.matched == 0

# Search for the newly created profile directory
- name: Check Firefox config folder again
  find:
    paths: "{{ ansible_user_dir }}/.mozilla/firefox"
    patterns: '^.*\.default-release'
    use_regex: true
    file_type: directory
  register: firefox_config_folder_search

# Set a fact with the profile directory path
- name: Set firefox folder name as fact
  set_fact:
    firefox_config_folder_path: "{{ firefox_config_folder_search.files[0].path }}"

# Add in the custom config
- name: Add in Firefox config
  copy:
    src: files/firefox/user.js
    dest: "{{ firefox_config_folder_path }}/user.js"
    owner: "{{ ansible_user_id }}"
    group: "{{ ansible_user_id }}"
    mode: 0644
  when: firefox_config_folder_path != 'false'

The play is idempotent so you can run it as many times as you want and it will continue to give you a nice green response.

I only ever use a single Firefox profile so I don't need to ensure different configs, but it could be extended to take this into account if you needed.

I later found some other software I use has this same config issue so I extracted the process start/kill tasks into a separate file

- name: Starting process {{ item }}
  shell:
    cmd: "{{ item }} &"
  changed_when: false

- name: Waiting for proces {{ item }} to start
  command: sleep 10s
  changed_when: false

- name: Killing process {{ item }}
  shell:
    cmd: kill $(pidof {{ item }})
  changed_when: false

Which can then be called with the following in your playbook

- name: Start/Kill myprogram to generate config
  include_tasks: start-kill-process.yml
  loop:
    - myprogram


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

The RAID

Posted: 2023-01-08 21:15:00 by Alasdair Keyes

Direct Link | RSS feed


After my failed drive on New Year's day, I ordered a new disk and rebuilt the array. Thankfully due to monthly checking by the OS, all the data on the three remaining drives was readable and the array is complete again.

I put the failed disk into another machine and ran the following badblocks command on it.

badblocks -o sdb_badblocks.txt -b 4096 -w -s /dev/sdb

I used the destructive test as the data was not needed now that the array was back to full strength. Incidentally, using a block size of 4096 over the default 1024 seemed to provide about a 2x-3x speed increase.

Even with that, the 2TB disk took just over 33 hours for a full write pass and a confirmation read pass.

At the end of it, a full write and read pass were managed with no errors reported. This is frustrating as mdadm had obviously detected a read error to reject the disk - this was logged in syslog.

I thought that maybe the bad sectors had been remapped by the firmware during the badblocks test, but checking the SMART stats again I saw that no errors are reported and also no re-allocation had been logged (ID# 5 below).

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   016    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   134   134   054    Pre-fail  Offline      -       103
  3 Spin_Up_Time            0x0007   168   168   024    Pre-fail  Always       -       342 (Average 311)
  4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       75
  5 Reallocated_Sector_Ct   0x0033   100   100   005    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   067    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   146   146   020    Pre-fail  Offline      -       29
  9 Power_On_Hours          0x0012   086   086   000    Old_age   Always       -       99078
 10 Spin_Retry_Count        0x0013   100   100   060    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       75
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       989
193 Load_Cycle_Count        0x0012   100   100   000    Old_age   Always       -       989
194 Temperature_Celsius     0x0002   200   200   000    Old_age   Always       -       30 (Min/Max 16/44)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   200   200   000    Old_age   Always       -       0

So I'm not sure why the error occurred. Maybe the controller is bad, the cable is dodgy or my server got hit by some stray cosmic rays somewhere and caused some kind of CRC error (yes, it can and does happen). The server has ECC memory so bitflips in the RAM should have been detected had they occurred.

Interestingly, this is the first failed disk I've had within a Linux MDADM array in over 20 years of running servers (I've had plenty of failed disks in Dell PERC controllers and whatever controllers Supermicro jam into their servers!). All previous arrays have been torn down before a disk failed.

As such this was also the first time I've had to rebuild an array. This particular RAID was running for over 11 years before this disk failed. For those interested, I followed this post by Redhat about the steps to take https://www.redhat.com/sysadmin/raid-drive-mdadm.

Should something similar happen again, I think I would run badblocks in non-destructive mode on the disk in situ, then if it passed push it back into the array for it to be rebuilt before I looked at buying a new disk.


If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz

© Alasdair Keyes

IT Consultancy Services

I'm now available for IT consultancy and software development services - Cloudee LTD.



Happy user of Digital Ocean (Affiliate link)


Version:master-8b74a9148e


Validate HTML 5