weeknotes for w/c 03/03/25

we did it. we pulled it off! codebar Festival Fringe was a resounding success, and the highlight of my week. it took three months of planning to pull this off, but it came together in the end. I'm currently curled up in bed watching Invincible, recovering from it all.

overall, the week went so well

• the weather is getting sunnier and less miserable, which is great!
• washed a really nice glass bottle I had laying about that was for sweet chili sauce. scraped off the label and refilled it!
• spent Monday in London
  • the local Greggs was shut on Monday. I was gutted because no sosig roll :(
  • went to Wellcome Library, and worked from there during the day
  • went to the Royal Opera House on Monday, saw Il trovatore. particularly enjoyed the anvil chorus. however, the captions cut out frequently and I was left with no clue what they were saying half the time
• hosted a lunchtime fireside chat with Helen Khor on Tuesday for codebar Festival, about how to navigate layoffs. it was a very difficult subject matter, but I feel Helen navigated it sensitively
• skipped the pub on Tuesday evening, as I had a nap after work. naps are so good
• the Internet went down on Tuesday night, so some of the homelab services were affected. that was a bit of an ordeal, and went to bed at 4am
  • accidentally upgraded Plane to v0.25.0, which broke my n8n integration. managed to fix it in short time, though. this taught me the hard way that when you're v0.x, you're allowed to have breaking changes with minor version bumps
  • when the Internet went down, I could still access the web when connected to Mullvad via Tailscale. that's super interesting, and I want to dig into it further.
  • this has got me thinking about a few things I can do to improve homelab resiliency. I think my next big project should fix a few of these things.
    • better monitoring. I get a constant stream of false negatives, perhaps because Uptime Kuma is a little sensitive. this has made the existing service monitoring ineffective, as I pretty much ignore all the downtime notifications I get. I should fix this so I start paying attention to them again.
    • want to introduce local DNS that routes requests to homelab-hosted services over the local network while I'm at home, similar to Willow's setup.
    • backups! written about this before, but I've been considering Backblaze. just need to pay them the money and get around to setting it up.
    • a proper service audit, to understand what services I'm running and what order to restore them in if things go down.
• critical service failover, for the main site and blog. could get another VPS running just those containers, with Postgres bidirectional logical replication to allow the Ghost database to be copied to the standby database (and to sync back to the locally hosted database once the incident is over). could stick all that behind Cloudflare Load Balancing.
• need to point Prometheus on my local servers over the local network, just in case of failures.
• in the longer term, investing in a redundant network connection might be useful. possibly mobile data?
• got inducted into the workplace's workshop on Wednesday. they have a bunch of cool equipment, including an electronics station, 3D printers, laser cutters and a bunch of other cool stuff.
  • I'm now qualified to use the fume hood, which is great! now have 53 other pieces of equipment to get inducted into.
  • I'm considering a workshop in my garden, so I can solder without choking on the fumes.
• worked from the Wellcome Library again on Thursday. heard about the Institute of Physics Members' Lounge on Caledonian Road, which wouold be pretty cool to work from
• went to the JSMonthly surprise event on Thursday evening at Foolproof! it was lovely to see Aris again, he's brilliant.
  • it was lovely to listen to talks about accessibility and AI running on local devices through the web. got a few new stickers, too! I closed out the event and managed to shamelessly plug codebar Festival Fringe.
• sadly can't make the next JSMonthly, as I'll be in Manchester for Queer in Tech. managed to book the trains for that this week, in fact!
• had a couple of friends come over on Saturday morning to help run codebar Festival Fringe. cooked them chicken thighs and mashed potatoes with a lovely sauce I made from scratch.
  • everyone loves the weighted blanket when they come around.
• pulled off codebar Festival Fringe this Saturday!
  • raised a cheeky £300 for charity. not bad for my first go. got plenty of useful feedback from Hugh Evans I'll use for the event postmortem blog post. also sent out a feedback form on Sunday!
  • Melissa was a real help organising the Fringe, specifically the comedy element. friends were also lovely with promotion in their social circles. I couldn't have done it without them.
  • met a bunch of interesting people, including Dr. Allensdottir, who focusses on space law. it's tangential to my work at Lumi in space lasers, and I've got brunch in the diary with her.
  • although it was a lot of work, I feel lifted by codebar Festival. the positive initial reception was great. considering doing it again, but I need to rest and recover