Posted: 2018-09-19 11:14:22 by Alasdair Keyes
Perl 5 is often seen as a dated, if not dead language, consigned to being hacked together by a sysad to keep some legacy platform from the 90s ticking over.
Sadly that part is true, but there is still life in the old dog yet, helped by the recent release of version 8 of the Mojolicious framework.
If you've not played with Perl but you have an interest, Mojolicious is a good entry point and I'd recommend it for any new projects. It's built in support for web-sockets makes it very suitable for modern back-end platforms.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-09-02 11:18:15 by Alasdair Keyes
During installation of Ubuntu Server 18.04 today I noticed that the user creation section had an option for configuring SSH public keys from Github, this is a very useful feature but I was intrigued as to how these were exposed. It turns out that you can use the following URLs to get the keys for a given user for Github and Gitlab
https://gitlab.com/<username>.keys
and https://github.com/<username>.keys
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-08-30 16:41:36 by Alasdair Keyes
After playing around with Wireguard today (of which I'll no doubt do a more fully featured post). I wrote a quite Nagios NRPE check. There's not much to check with Wireguard, but it queries the wg show
output for the number of interfaces and peers. Full arguments are in the README. Let me know if you have any suggestions for future behaviour.
https://gitlab.com/alasdairkeyes/nagios-plugin-check_wireguard
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-08-29 14:48:04 by Alasdair Keyes
Since HTTP/2 was implemented in NGINX, I've wanted to use it on my sites. Unfortunately due to one of my VPS providers not providing a Debian 9 image, I was stuck on Debian 8 with an NGINX version that didn't support it.
Now that I've upgraded all my machines to Debian 9 I have finally updated my sites to make use of HTTP2. Combined with the upgrade to PHP7.x at the same time it should allow some performance increase for my sites.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-07-19 21:46:48 by Alasdair Keyes
At some point in the recent past Linux Mint 18.3/Ubuntu 16.04 Xenial had updated lxc
containers to a more recent version and when running lxc-ls
I was greeted with the following error for all of my containers.
# lxc-ls
Failed to load config for mydevcontainer1
Failed to load config for mydevcontainer2
Failed to load config for mydevcontainer3
Failed to load config for mydevcontainer4
It appears support for an older container config version has been discontinued and it was unable to read the old versions, a bit of Googling found the answer was to use lxc-update-config
helper utility.
# lxc-update-config -c /var/lib/lxc/mydevcontainer/config
# lxc-start -n mydevcontainer
#
For those with a lot of containers, the following one liner will resolve them all for you in one fell swoop
lxc-ls 2>&1 | grep "Failed to load config for" | awk '{print $NF}' | xargs -i lxc-update-config -c /var/lib/lxc/'{}'/config
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-07-08 16:54:20 by Alasdair Keyes
I've had a Gitlab account for a couple of years because of their facility for free Private repos and since then I have pondered moving all my Public Github repos across but never had the inspiration. The main reason not to was because Github was the de-facto Source Control site, so when dealing with non-tech people like recruiters, it was easier to just point people at Github rather than explaining the lesser known (at the time) Gitlab.
Now with the acquisition of Github by Microsoft, I've got the inspiration. With the exodus of many repos to Gitlab, they now have a much greater reputation outside the tech-circle so my code has now been moved.
The Github projects still exist but just have a README.md directing people to Gitlab.
https://gitlab.com/alasdairkeyes
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-05-31 08:01:22 by Alasdair Keyes
For such a versatile language, Perl has no good built-in way of running an external commands.
You can obviously use exec()
, system()
or even the dreaded back ticks ``
. But often when running commands you require different things. Maybe you just need an exit code, so system()
is good. If you're looking to capture output, exec()
or backticks are more suited for the job, but then escaping input and using pipes or redirects then becomes a nightmare.
Last year worked on a project that required a large amount of systems integration and calling binaries, to run tasks and process output so I had to find a reliable and useful method. Mixing system()
, exec()
was possible but as I would be handing the code off to the customer, having a single way of running commands is a lot cleaner.
I came across https://metacpan.org/pod/Capture::Tiny. It provides a clean interface for capturing output from both Perl code and running system binaries.
#!/usr/bin/env perl
use strict;
use warnings;
use Capture::Tiny ':all';
use v5.20;
my ($stdout, $stderr, $exit_code) = capture {
system("ls", "/var");
};
say "STDOUT\n$stdout";
say "STDERR\n$stderr";
say "EXITCODE: $exit_code";
STDOUT
backups
cache
games
lib
local
lock
log
mail
opt
run
spool
tmp
STDERR
EXITCODE: 0
The only flaw seems to be that it doesn't handle the capturing of output long running processes that will spit out text slowly. For that you will want to use something like pump on https://metacpan.org/pod/IPC::Run
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-05-30 17:07:47 by Alasdair Keyes
The joy of contracting is that I often get to work on brand new code-bases with some regularity. The down side is exactly the same.
Over a number of contracts I've noticed a number of things that I've suggested to teams that make their development much easier.
If you work for a fair-sized business or a 'tech' company you can stop reading here, you will look at these suggestions, scoff, and wonder why they need making. However there are a lot of companies that are not 'tech' focused. Their website functionality was bolted onto an existing business that pre-dates the web and the first site was a few scripts written by the most tech-savvy employee they had at the time and grew organically from there. Yes, these code-bases are often a mess and constant tight deadlines for new features and product means it always stays that way and doesn't get improved, but that's reason not to try. It's usually the devs that have to work with the code that get most jaded with the situation, having to work with the code day-in and day-out.
These suggestions are mainly based around PHP, but they can be easily adapted for any other language.
This might seem like a no-brainer, but I still see teams without Version control (or still stuck with CVS/SVN). Creating an account through Gitlab is highly recommended, not only do you get their git system, they also have free CI/CD pipelines to allow automated testing which will come in handy if you wish to use these later.
We've all seen code like this, or if you're unlucky with even less line breaks or line breaks in crazy places
public function fib($n) {if ($n < 0) { return NULL; }
elseif ($n === 0) { return 0; } elseif ($n === 1 || $n === 2) { return 1; }
else { return fibonacci($n-1) + fibonacci($n-2); }}
The code's purpose is not easily understood and the name fib
doesn't instantly describe what the function is doing.
PSR-2 is the PHP Framework Interoprability Group's approach which has become the defacto standard for PHP and will make your code much nicer to work with.
Along with nice naming you can turn the above into...
public function fibonacci($n) {
if ($n < 0) {
return NULL;
} elseif ($n === 0) {
return 0;
} elseif ($n === 1 || $n === 2) {
return 1;
} else {
return fibonacci($n-1) + fibonacci($n-2);
}
}
You don't have to strictly adhere to a well-known standard, but using a consistent, clean formatting approach can make your life just a little bit easier.
You can look at doing it in one fell swoop with a tool such as PHP Coding Standards Fixer or do a gradual merge whereby all files that are edited with new code get reformatted before any changes and committed to the repo. Tools such as PHPStorm also have a formatting feature available using CTRL+ALT+L
Each language has it's own documentation format PHPDoc for PHP, Perldoc for Perl etc. What you use isn't so much the key as just using it. A quick one line introduction to a class/function plus documentation on the arguments expected and return values will make your code much more pleasant for people unfamiliar with it.
Again, a rolling adoption is easy and fairly hassle free. Write the required docs for new functions, and then as you edit existing ones, back-fill your documentation. In addition if you use code reviews, get people to check documentation updates as well as the code being committed.
The chances are if you have not been writing tests from the start, it will not be easy to start introducing tests to old code. The primary reason is probably you won't have written your code in a manner to allow easy testing, you won't be injecting dependencies and your functions will be performing too much work to create concise tests.
Don't let this put you off. Simple tests can be put in place quite easily for even the most attrocious code base.
Firstly, if your team is unfamiliar, take a morning to read up on the different types of tests commonly used (unit, integration, functional, Behaviour etc), the benefits they provide and when you are likely to use them. There are plenty of pages and Youtube videos available via Google to get you started.
Secondly, look through your code base for simple functions that are able to have unit tests created for them. These might be few and far between to start with, but once the tests are written, you can get a little more piece of mind that some future commit is less likely to break functionality without anyone noticing.
If you use a common framework, most of these provide a customised test suite to enable rapid test development for WebApp functionality. A simple, quick and useful test could be to ensure that all access to login restricted URLs returns a suitable message and/or HTTP status code along with a redirect to a login page. This can then perhaps be extended to test valid/invalid login messages and redirects. It's a small step, but you're heading in the right direction.
Testing is a huge subject and you could easily get drawn down a rabbit hole, however, with just a few test, you are already more certain of stopping some errors creeping into your code.
This isn't an exhaustive list, if you find that you are in need of implementing some of these you will likely find that there are lots more changes required in your development processes, but these will allow you to gain a lot of benefit without tying up devs for days or months.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-05-28 15:41:44 by Alasdair Keyes
Last week I finished rolling out my new Debian 9 "Stretch" machines and noticed some peculiar behaviour in vim. Namely, my tabs were being kept as tabs and not converted automatically to spaces. We'll ignore the old "tabs vs. spaces" debate for the moment but I was struggling to see why my tried and tested vim config wasn't working.
My .vimrc
file had the following lines
" Set tabs to 4 spaces
set shiftwidth=4
set ts=4
set expandtab
" Allow simple pasting and add ruler/linecounts
set paste
After some hunting I found the following article explaining the issue https://bugs.launchpad.net/ubuntu/+source/vim/+bug/1576583
Debian 9 moves to vim version 8.0 and it seems that the behaviour of paste has changed so that it resets expandtab
. All that's required is to move the set paste
line before expandtab
and you're back in business. Hopefully this might help someone else who has seen the same issues.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Posted: 2018-04-24 09:30:10 by Alasdair Keyes
Until recently, the GnuPG section of my site which lists my GPG fingerprint had a link to keys.gnupg.net for visitors to make some verification of my key.
I accidentally clicked this link a few days ago and noticed that I was redirected to https://analytics.sumptuouscapital.com/
instead of the expected https://keys.gnupg.net
website. I checked with Mike and he was seeing the correct site.
This looked interesting, it could be some misconfiguration or potentially something more nefarious like a DNS poisoning.
I dug into it a little and it looks like gnupgp.net operates a round-robin DNS setup for it's web server cluster with 9 hosts.
$ host -t A keys.gnupg.net
keys.gnupg.net is an alias for hkps.pool.sks-keyservers.net.
hkps.pool.sks-keyservers.net has address 193.224.163.43
hkps.pool.sks-keyservers.net has address 193.164.133.100
hkps.pool.sks-keyservers.net has address 176.9.147.41
hkps.pool.sks-keyservers.net has address 192.94.109.73
hkps.pool.sks-keyservers.net has address 51.15.53.138
hkps.pool.sks-keyservers.net has address 216.66.15.2
hkps.pool.sks-keyservers.net has address 68.187.0.77
hkps.pool.sks-keyservers.net has address 92.43.111.21
hkps.pool.sks-keyservers.net has address 37.191.226.104
I wrote a small script to query each individual IP for the keys.gnupg.net
website, the result was
On the plus side, it doesn't look to be anything nefarious, just lack of maintenance and competence. It looks as though the GnuPG keys webs server setup is really broken, I have no idea how long this has been broken in this way, but it doesn't scream 'secure'.
As such, I've removed the link from my site and I now just use pgp.mit.edu and I suggest you stop using it too.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
© Alasdair Keyes
I'm now available for IT consultancy and software development services - Cloudee LTD.
Happy user of Digital Ocean (Affiliate link)