<  | >

Hacker's Diary

A rough account of what I did with Emacs recently.

December 31
OpenHAB migration: holy crap is this unpleasant. Naive backup/restore seems to trash the destination such that it won't even start, which somewhat undermines the whole "java: write once, run anywhere" shtick - I suspect there's at least some JNDI tucked away in there. Options for recovering seem limited: blowing away the cache (the cache) appears to nuke all my add-ons, which is at least part of what I'm trying to avoid. Using the restore script's optional arguments ... crashes the restore script because it's trying to copy files to a directory that doesn't exist.

Ok, so fixing that... doesn't restore things I'd expect to be restored, like the aforementioned addons list.

Manually restoring the addons.cfg seems to have triggered downloads of the missing bits and pieces; I still had to recreate my database connection, however.

Also managed to consume all memory running pip to set up one of the auxilliary programs I use with OpenHAB, so that was a bit of a laugh. Ha. Ha. Ha. Anyway, it all seems to be working now, so I can start cleaning up the old install. I'm not 100% convinced the RPi is muscly enough to handle the load, but we'll see.

A Haunting In Venice: mostly good, but did not need to spooky bit at the end. Mais oui, the little grey cells are still suffering mildly from the hallucinations, n'est-ce pas? Sigh, if only it'd actually been played that way. But no, the ghost drags the murder off the building in front of Poirot. Reminds me of the time they did something similar in Midsommer Murders: that wasn't particuarly a great idea, either.

December 30
As expected, The Blacklist did not kill off the lead character mid-season. Some of what's going on is silly, some is fun.

Speaking of silly: spent some time trying to build an old piece of software in support of my OpenHAB migration efforts, and eventually figured out that my attempts to set CFLAGS on the make command line were causing it to discard all the CFLAGS - including -c, which is sort of essential for the Makefile-as-written to function. Once I'd figured that out and swore a bit, the compile worked. I haven't checked if it's produced a viable binary yet, mind you.

Ok, it works, but also slightly surprising: I plugged in my spare controller, which I'm sure I'd last used configured as a secondary, and it immediately chatted to everything on the Z-Wave network as if it were primary. Er. That doesn't seem right.

December 29
A day of hardware failure. The laptop once again did the slow failure thing, and in the meantime the powerline ethernet did a thing it hasn't done since I upgraded my router: lost connectivity between nodes, dropping my jump box off the network. Said jump box is headless so I initially tried just rebooting it which obviously didn't work; I had to root out keyboard, monitor and mouse to log into it at which point I realised what the problem was.

And then OpenHAB didn't want to start, but that's been a long-standing problem (as far as I can tell, because of a race condition in service startup) and I really need to get on with my plan to ship the system over to a raspberry Pi.

December 28
The season closer for Slow Horses was pretty brutal in spots. It echos Herron's writing to some extent, but I think there was just a smidge of wallowing in the TV production. Alas, as previously mentioned, it's been a very short season again - it's only six episodes per - so now we wait presumably an entire year for another burst of activity.

Reflecting on the fact that part of what I enjoy about hacking about on random software toys is the act of making a thing I want. I have a webscraping toy here that serves an entertainment-related purpose, and I wanted a means of visualising the freshness of the data, so I cobbled together something that did exactly that; I'm looking at it now thinking, "I made this". I can't do this for physical stuff, for the most part: I'm kinda inept when it comes to manual activity, and too impatient to practice until I'm good at it. Software allows me to do just enough to make it work and then refine that as and when required.

Got a COVID booster and a flu vaccine today. Not a moment too soon, it seems, as there's apparently a surge of cases of both presenting at hospitals. I am, of course, going to hurt for the next several hours.

December 27
404s from ABRP turns out to have been a combination of me posting not-quite-valid URLs (I'd accidentally included a double-slash in the path somewhere) and ABRP changing some internal infrastructure that no longer accepts such URLs. I've also added timestamping to my log messages so next time I see a failure I'll know when it happened...

lsp-mode is still finding the wrong python interpreter. What I'm trying to do with it is have it, on client invocation, traverse the directory tree upwards from the current file location looking for either bin/python or bin/.venv/python; if it finds such, it prepends the location into a buffer-local version of exec-path, which should by my reckoning make the Pyright wrapper code pick up the correct Python executable. So far, it seems to have worked intermittently, and it also seems to possibly be setting up a multi-root workspace when I don't really want it to do that.

Hmm. One of the things that wasn't working on the office setup is down to it referencing a variable that was introduced in a newer version of Emacs than I'm running there. That's annoying because upgrading the version isn't easy for handwaving reasons.

No progress made on OpenHAB today because I had a few too many other things to deal with.

Interesting variant on the failing macOS issue: Safari stopped working. In attempting to diagnose and recover, I wound up with the machine in a state where the only thing I could do was hard-boot it. This was after I'd successfully logged out and logged in again, so whatever's happening is happening at the system level rather than at the user level. Given this machine has been incrementally upgraded from one OS revision to the next over the course of 12 years, it's entirely possible there's something running around in the lower levels of the OS that's Just Not Right (such as whatever's providing me with no less than 12 network interfaces on a piece of hardware with one ethernet port and one WiFi radio).

Installing a Perl language server on macOS was a hoot... I encountered a comment to the effect that macOS Perl is for Apple, not for you, but the linked source didn't expand on that comment so I don't really know what to make of it. I was able to get things running through setting CPATH in the shell I used to invoke CPAN.

December 26
lsp-mode tinkering continues. I've learned a few small things about it but not quite enough to write down a specific guide (for myself or others) to dealing with the various environments I'd like to use it in. Some of the documentation is either absent or opaque, and some lightweight browsing of the code hasn't helped hugely as it's also a little on the opaque side - plus, truth be told, it's a long time since I worked with elisp on a regular basis and I was never a skilled practitioner with it. A lot of it is readable from the perspective of "it's an ALGOL-derived language" but some of the specifics of syntax are things that appear to have arrived after I last spent non-trivial time in the innards of the BBDb.

Beyond that, even when it's working there are still a few things I haven't quite grasped, like how to force it to report on what's wrong with the current line when the automated tool-tip/hover-help/whatever decides it's not playing ball. It's a little frustrating having it mark a line as in need of attention without being able to get it to tell me why.

On the plus side, it has picked up a couple of things in Python scripts that I've been using for a while that flake8 didn't spot, so that's pretty good.

OpenHAB upgrade! The service installation on macOS is, as ever, still broken. I'm moving my install to a Raspberry Pi instead.

Theory of moving OpenHAB installation: Unfortunately it seems there are some hardcoded path names in the config files, and because I was moving it from macOS (everything in /Application/OpenHAB) to Linux (files scattered across at least four directories), some of the hardcoded pathnames were looking in the wrong place. Tsk, this seems like the sort of thing an application-specific restore should handle.

Then there's the add-on configurations which contain refs to old locations, and the various hacks I've piled on top, and the database for persistence, and the MQTT server...

While digging into this I noticed that the A Better Route Planner live data API was returning 404s. That's annoying. More so that I don't know quite when it started because I neglected to add timestamps to my logs but the app claims it got live data from me 4 hours ago.

Anyway. I've confirmed that most of the things work except the one thing I can't test without flipping the servers around, which is the Z-Wave dongle. I guess I'll have a look at that tomorrow.

Oh my. Doctor Who and the Daleks somehow manages to look cheaper than any of the TV iterations I've seen. It's ... quite terrible.

December 25
Seasons etc. etc. etc. I have some nice new clothes and made a pretty decent spread for dinner.

December 24
Death on the Nile was another easy watch. I've seen the David Suchet version, so I was broadly familiar with the plot despite some alterations made to it for this version. Beautifully shot and a pretty decent cast.

December 23
Indiana Jones and the Dial of Destiny was an easy watch. Nothing terribly surprising, proper use of Chekov's Gun allowing a certain amount of guessing what was going to happen, silliness, a few age jokes, callbacks to previous movies. Everything you'd expect from an Indy movie, really.

December 22
It has been a hectic week.

Reacher: continues to be excellent. Slow Horses: ditto. The Blacklist: the "let's put Reddington in jail" thing seems to be wearing a bit thin, to be honest. I liked that they wrung a season out of "What if Reddington lost all his money", but they don't seem to be quite so capable with this plotline and Liz keeps demonstrating a remarkable lack of aptitude for her chosen profession.

December 16
The Equalizer 3 was so-so. A lot slower than the other movies, the big finale was... somewhat underwhelming, and there seemed to be an awful lot of loose ends just left lying around. Like, what happened to the chief of police and his misfortunate hand? What about the café girl? Anyway. It passed the time and wasn't terrible, so I guess there's that.

December 15
Oh hey, a new season of Reacher. All these new seasons would be great if they didn't last, at best, for 2 - 3 months before leaving you waiting the rest of the year for the next one.

Laptop got into a familiar state: apps crashing, repeated popup from something to do with Safari, errors from accountsd because it no longer has access to the keychain. This is what eventually caused me to shut down my LSP experiments last time, since my LSP tinkering was strongly correlated with the Mac getting into this state; I think it's either something timing out or something getting killed off due to memory pressure, and once it happens it's unrecoverable and I end up doing a hard shutdown to get back to some semblance of working order. It's really annoying. The thing is, I've got 16GB of memory and a SSD, so it's irritating that this is somehow not enough for whatever is doing this.

December 13
Continuing adventures in LSP, let's go check out these Python servers: Now, of course I can't remember which one I used before but it looks like it's going to be a toss-up between Pyright and Python-language-server. Of those two, Pyright was last updated 2 hours ago (!) while Python-language-server was last updated in 2022, so I guess Pyright is actually being maintained.

So that's package-install lsp-pyright and let's see what happens.

Brief confusion on account of having installed django-mode, which apparently took precedence over python-mode and I didn't notice despite it being right there on the modeline.

Ah, ok, this is the one I used before. And was confounded by, somewhat. The essential problem was getting it to use the right Python interpreter; it seems to have some sort of virtual env support, but the configuration for same is somewhat opaque to me. Reading through it again, I'm still confused. Basically, what I'd like to do is to say "use $envroot/bin/python as the python executable and let everything else fall from that". Instead I have options to define "execution environments" and "venv paths" and "venv directories", all of which make assumptions about how my environment is set up that ... are incorrect. And I recall debugging this the last time around and discovering there seemed to be an undocumented (or not obviously documented) hook into a project-management mode that sort-of made things work. Hrm.

Ok, after some tinkering around and eventually looking at the elisp source, setting lsp-pyright-venv-path to the virtualenv root seems to do the trick. Curiously, this variable is not buffer-local by default. I can work with this, I guess: hook the lsp-mode startup for a buffer and dynamically set this variable to point at the correct directory. There appears to be less rigid support, but it appears to be tacitly dependent on my using directories called venv or .venv, which I don't. Which may be something I'm doing wrong? I have no idea. Enough I guess that I have this working for one project for now.

(I'm thinking my clever hook function should probably start by checking for "bin/python", then falling back to some system python, since that's approximately how my various toys are laid out.)

December 12
Spent far too long reinstalling a Windows laptop, hampered by the fact that the reinstalled OS comes with a "sleep laptop after 30 minutes" setting, and Windows apparently prefers to let the laptop sleep rather than continue with an unattended install that requires multiple reboots. Painful. I still don't know what brought about the need for the reboot, because Windows diagnostics are something of a closed book to me; it's possible it was the recently-announced UEFI bug, but it's equally possible that a bad update from Microsoft broke it. There definitely looks to have been a situation where a process took too long and timed out, and whatever was watching or controlling that process didn't cater for this possibility. Anyway, all back now, via a "Windows Reset".

Part of my attempts to diagnose the problem before just paving it involved browsing a collection of CD and DVD ROMs for tools. Wow. Those things are so low-capacity now, by comparison to any half-assed USB drive. 650MB! 4.7GB! Tiny things. And not all that old, really.

(For calibration: my first computer was able to store about 600K on a 90-minute cassette tape, and the first computer I owned with onboard permanent storage had a pair of 20MB pre-IDE (MFM?) drives.)

December 11
Replacing a few of the tools I use to manage this website with Python versions. The current toolset is basically a big ugly shellscript that calls a bunch of Perl scripts and then rsyncs to the site (no I do not want an active CMS on my little web server, why do you ask?) so I'm moving the functionality of those perl scripts into the python script I wrote to do crawl-and-update work across the site (e.g. convert all links to IMDb to https, for example).

December 10
The new season of Doctor Who is officially bonkers. And I'm sure all the lovely people who complained about Rose's character will have a field day with the new Doctor, not limited to his state of undress.

December 9
I have been tinkering with lsp-mode on and off during the week. I got it freshly set up at work, now that I know how (nuke the site from orbit, remember?), and then discovered that quirks of work's development mode mean it doesn't quite work out of the box so I need to hack around it a bit. I still haven't opened the "which Python LSP server is best" can of worms, either. And I had a bit of a fight with the Ruby LSP server: there's, again, a choice of a half-dozen with no clear recommendations, and they're all installed via gem which for handwaving reasons is at minimum annoying in the office environment. Now, where lsp-mode is working, it's working pretty well, but I definitely need to read more about operating it.

I did eventually figure out how to tell it to shut up about languages for which no server exists. Actually, even without reading any further, browsing through M-x customize-group RET lsp-mode RET was helpful.

December 8
Slow Horses continues to be epic. Also it looks like Prime Video opened up more of The Blacklist, so we've launched into season 6. And somewhere in there we wrapped Season 2 of Cold Case, which was slightly disappointing in as much as Revisiting Bad Guy turned out to be fairly mundane in the end and was somewhat trivially dealt with despite Lily deciding to do the old "Go Visit The Haunted House On My Own" trope.

December 3
I'm inclined to take a crack at setting up emacs-lsp again, if only because some recent TypeScript work has had me occasionally reaching for Visual Studio Code to do some stuff and then spending the next ten minutes swearing because it doesn't have Emacs keybindings and I'm apparently hardwired to use Ctrl-a and Ctrl-e a lot, both of which are intrusive to current activities in VSC. This led to me reading through my entire .emacs (which, for historical reasons, is in fact .xemacs/init.el) for the first time in a long time and wondering just how much of it is still relevant - never mind feature support for Emacs 18.59, which hasn't even been compilable on modern systems for some years now, there's also support in there for things I no longer use, like the VM mailer, BBDB, and GNUS. Oh, and ange-ftp for that really early Internet feeling.

Keeping notes this time, too, and following the Vanilla Emacs instructions from the site since I'm on macOS and not using one of those fancy emacsen.

Step one, delete all traces of lsp I can identify and restart emacs: done.

Step two, package-install lsp-mode: done.

Step three, and this is where I have to think about it a bit, install a bunch of additional packages. Some of these resulted in poor use of limited screen real-estate; I suspect they're built by and for people who spend all their time not using laptop screens. I guess I can try 'em out and see what works and what doesn't. So, package-install lsp-ui, flycheck (already installed?), company-mode (same), lsp-treemacs, helm-lsp (missing dependency popup.el), lsp-ivy (package tarball not found), dap-mode: well, that was a partial success. M-x package-refresh, and now helm-lsp is missing a different dependency, but I managed to get lsp-ivy. Guess I'll do without helm-lsp, which in any case seems to overlap lsp-ivy?

Step four... ok. This is where I have to glue things together to make them work. The instructions suggest setting up for all programming languages by using prog-mode-hook so I guess I'll use that, plus defer loading and what not: (require 'lsp-mode) and (add-hook 'prog-mode-hook #'lsp-deferred)

Step five, restart Emacs: done.

Step six, open a TypeScript file, get prompted to install a language server, get prompted as to whether I want LSP to be active for this file, and off we go. Gritch gritch, it didn't bury the Comint buffer from the install.

Ok, so it's doing various syntax checks, but it's not importing the library definitions, which is partly why I wanted this (saves me trawling through pages of CDK documentation, for example). So I guess I need more configuration. Cunningly, there is lsp-doctor which tells me I need to mess with read-process-output-max, something that the install process neglected to mention. It looks like I'd also jacked up the value of gc-cons-threshold to something silly, so this time I'm paying attention to guidance to merely double it until performance improvements stop. Restart and try again.

There's a section in there on using plists vs. hash-tables, but it gives no indication what the tradeoffs are other than that plists are better - so why not enable that by default?

Step seven, did some tuning and it's still not helping me with my AWS CDK completion.

Ok, so this seems to be a fight between what the compile-chain is willing to work with and what the LSP is complaining about. So for example import { aws_ec2 } from 'aws-cdk-lib' compiles just fine, but LSP is insisting that aws_ec2 is not exported from aws-cdk-lib, and flipping it around to import * as aws_ec2 from 'aws-cdk-lib/aws-ec2' complains that aws-cdk-lib/aws-ec2/index.d.ts is not a module, neither of which seem to inhibit actual complilation / execution.

Checking the Languages page of the lsp-mode docs, I discover I chose the one TypeScript language server which was unmaintained. Of course. Is there a lsp-uninstall-server? No there is not. Kill all the buffers, quit Emacs, nuke the cache directory where it downloaded the server because it's the only thing in it, and start editing again.

Installing the recommended server: crashes on startup without showing me the error. (What step am I on now? Is this still part of one of the previous steps?). Digging around in the buffers I find "error: unknown option '--tsserver-path'". Trawling in the package cache turns this up in lsp-javascript.el and the changelog for the language server itself and appears to be a going concern? For good measure, the lsp log appears to be trying to pass /usr/local/bin/tsserver as the server path, which doesn't seem right in any case. Excellent.

Digging a bit more, as best I can tell the version of the NPM wrapper around the typescript server it's installed doesn't support the --tsserver-path. So I guess I'll see if I can prune that out.

This is getting tedious. Remove the cached files and try again. It's using npm to install the typescript-language-server and not getting the right version, as best I can tell, but there's no obvious indication why that might be.

Ah, I see my problem. I foolishly chose to install from the Stable MELPA repository rather than the, I guess, YOLO repo. Sigh. Revisiting step three to check package versions... right, well, that went great assuming I can ignore the various compiler warnings. Let's restart Emacs again, shall we?

Off to a flying start with Error running timer: (void-variable lsp-ada-project-file). Wonder if that's some sort of hangover from previous adventures in this space, given noone ever seems to provide a cleanup process?1 Apparently this is a reported bug and the fix is take off and nuke the site from orbit, it's the only way to be sure. Delightful. Exits Emacs for the tenth time, deletes entire directory of cached crud, starts over.

FINALLY. I actually had to do the exit-cleanup-restart process twice, because evidently I missed something, but now my TypeScript editing is working as intended. I'm going to hold off doing anything silly like customising it for the moment.

Ok, that's kinda funny. The one language definitely not supported by lsp-mode is ... Emacs Lisp. And you can just imagine the arguments being used by People Who Know Better Than You about why this is The Right Thing.

Tomorrow: figuring out which of the six Python LSPs is the best one for my use-case, given that last time I got lost in a maze of Microsoft "documentation" that didn't make sense to me even in the context of the editor for which it was written.

After we watched The Ninth Gate a couple of months back, I picked up a paperback copy of the source novel (The Dumas Club) and reread it. I'd forgotten details of the original story and quite how much was lost by the excision of the eponymous club from the movie.

1. Turns out there's a package-delete. I gave up after looking for package-remove and package-uninstall.

December 2
Also a new season of Doctor Who started last week. So that's a bit of fun.

December 1
Oh hey new season of Slow Horses. Let's be having you, then.

previous month | current month | next month

Another year closing out.