Skip to content

New Infrastructure

Hello reader, welcome to 2025!

I've heard the saying "new year, new me" before, but quite frankly, I feel like "new year, new devops solution to a simple bash script" is more fitting. This is just a short blog post to explain how this blog is actually hosted! Very meta.

Pre-2025 & Codeberg Pages

Prior to 2025, I didn't really take the blog idea seriously. I just wanted somewhere to dump my long-form posts that I was previously writing up on the fediverse, somewhere that would always be easy to locate and view.

So, I set up a repo on codeberg.org, and made use of their https://codeberg.pages service. This was great, as I could enjoy the luxury of just pushing the changes to my blog to the repo, getting a very simple build CI workflow that would build and push my changes to the pages branch, which would then be automatically deployed to my blog, located at https://blog.nexy7574.co.uk.

I enjoyed this luxury for several months, however, during this time, I noticed that codeberg pages proved to be more and more unreliable. Don't get me wrong, it's not THAT unreliable - my status monitor puts it at between 96% and 98% uptime monthly, which for most people is absolutely fine. But, I mentioned earlier that I wanted my blog to be always accessible, so I sought out an alternative.

GitHub Pages?

Yes, GitHub pages are reliable and fast. However, I am not comfortable putting something as personal as my blog on GitHub (who are now owned by Microsoft, and have one of the largest self-sufficient AI tools that openly scrapes their own user-generated content), as I feel that I am in theory limited in what content I am allowed to publish. As I frequently sprinkle my opinions and other possibly "controversial" topics into my works, I do not want to risk what is rapidly becoming my professional portfolio being tainted, or even being redacted by someone who reported the repository.

Plus, I trust my own hardware more. I do still mirror to git.gay and codeberg.org (more below), but frankly I trust a charity and a queer public service group than a faceless corporate entity that has more money than I can imagine.

Brief migration to pages.gay

I had heard a lot about git.gay online before, and after signing up for their instance in late 2024, I found their server was so much faster than reliable than codeberg's (it's important to note that at this time, Codeberg was being hammered by state-sponsored AI scrapers, who were sapping all of their bandwidth. Not their fault!).

So, I migrated some repos over to git.gay, including my blog, and set up the blog with https://pages.gay. Luckily, because they were using very similar software, I had a few minimal changes to make (namely, migrate DNS records), and I was up and running.

However, nothing lasts forever. I started getting downtime notifications regarding my blog in particular, and now pages.gay stands at about 97% uptime, which is still great, but also...

On my status page, I also monitor my own uptime (the monitor is hosted on a cloud server). Probing nexy7574.co.uk with the same frequency and settings as pages.gay and codeberg.page, and found that I myself have a 99.89% uptime, on average, monthly.

Hosting my own pages server

I decided I should host my own pages server, however, after taking a look at both the codeberg-pages server, and git.gay's pages server, I realised that I would need an extra IPv4 address. There's an issue here - I can't afford that. Hetzner wanted to charge me a non-insignificant amount of money for an additional IPv4 address, and I can't call up my ISP and ask for an extra IP for a domestic connection (they probably offer it for commercial connections, but like, I'm not a business lol), so I was stuck between a rock and a hard place. I tried to work around the limitation of this software, but eventually, I just outright gave up.

Early 2025 - a home run!

Eventually, I got sick of several daily notifications regarding both codeberg pages and pages.gay having outages, and decided to move to my own infrastructure for real this time.

I spent a lot of today (and yesterday) working out how I would do this, and decided, I could just use rsync to copy over my built blog files over to my server, which is already serving most of my content with Caddy (which has a built-in featureful file-server, and automatic TLS).

A plan was coming together. I opened an SSH connection to my reverse proxy, added a new system account, generated an SSH key, granted my build CI worker VM outgoing SSH connections to my reverse proxy in the firewall, added these lines to my workflow, popped the private key into the workers secrets, populated the variables appropriately, and... pushed a commit (well, 8 of them)!

The pre-existing build job needed minimal adjustments, it just needed to create an artifact that the new publish job could download and sling over to my server, and then I added the following snippet to my caddy config:

nexy.blog.Caddyfile
1
2
3
4
5
6
7
8
nexy.blog,
www.nexy.blog,
blog.nexy7574.co.uk,
blog.i-am.nexus {
        root * /srv/http/blog/deployed/
        encode zstd gzip
        file_server
}

And that was it! Changed the DNS records over on my registrar, and here we are.

This blog post is going to be the first post not only of 2025, but also created entirely within my house, built in my house, published in my house, and then served from my house! Talk about fresh out of the press!

Step by Step buide

Update (2025-01-23):

I've had a few people now express interest in this deployment method, as it's quite simple. Also, since using this for my blog, I've also adapted it to delpoy my main website, which actually is irrelevant here, but I guess it shows you how you can deploy a completely static site without a build step and whatnot.

In case you were wonding how I did this, here's a shortened breakdown of the steps I took, including details around how I secured this.

Do not grant unknown workers access to your hardware!

By the nature of this deployment method, you are handing over a private SSH key to a remote CI worker. For me, this is secure, as the worker I use is hosted on my own hardware, along with the git server itself. In theory, the SSH key I use never actually leaves my LAN.

However, if you were to use something like GitHub Actions, you are potentially handing over passwordless authentication to an unknown number of faceless entities. Remember, CI secrets are unencrypted and in theory can be read by the service provider at any time.

Set up the firewall

I run my CI workers in an isolated virtual machine running on my Proxmox node, which by nature of being isolated, is rather secure, and can't make any outgoing network requests to anything ports other than port 80 and 443. However, in order for this deployment method to work, I needed to allow the build VM access to my reverse proxy LXC.

Rather than just circumventing the firewall protections, I instead just added a rule to the firewall: ALLOW FROM internal_bridge TO reverse_proxy_ip PORT 22. This means that my build VM still cannot accidentally SSH into anywhere else on my network, only my reverse proxy, meaning the blast radius in the worst case scenario is limited to one, maybe two containers at most. Compartmentalisation ❤️

Adding a dedicated user account

In line with this security, I also added an additional dedicated account to my reverse proxy, as to reduce the privilage of this workflow. Since the account only needs to serve files, it doesn't really need to access anything else in the system, I ran the below command to create an account called blog with its home located at /srv/http/blog.

Why put it in srv?

No special reason other than /var/www is already used by all of my other static files. It feels important to make the distinction in /srv/http/blog, because to me, the directories in /var/www need to be manually updated. However, putting it in /srv/http means that, while it's still obviously related to the web server, I also know that it is special, and in thise case, managed externally.

You could quite easily set the account's home to /var/www/blog, /home/blog, so on so forth.

# Ubuntu 24.04 LTS, as root
adduser --home /srv/http/blog --disabled-password blog

I then needed to add Caddy's user to the user group created for the blog, so that it can read (but not modify) the files generated by the blog worker:

# Still as root
usermod -aG blog caddy

I then rebooted the entire LXC, however, you might be able to get away with restarting a few services. Either way, I knew restarting the LXC entirely would make sure that the group change had been applied.

Generating the SSH key

I had now set up a new user, but I needed to generate the SSH key to be used by the CI worker.

This was actually the simplest part, all I had to do was run ssh-keygen:

# Ubuntu 24.04 LTS, as blog:
ssh-keygen -t ed25519

There was one additional step after this that I actually missed the first time around - adding the newly generated public key to the authorized_keys file:

cp $HOME/.ssh/id_ed25519.pub $HOME/.ssh/authorized_keys

And that's it! The SSH key was now ready to be used. I made sure to cat $HOME/.ssh/id_ed25519 and copied the private key text to my clipboard for later.

Setting the SSH variables and secrets.

If you took a look at the workflow file, you will have noticed the following variables were used:

  • SSH_USER - the username to log in with. In my case, blog.
  • SSH_HOST - the hostname or IP address to connect to. In my case, this was just my reverse proxy's LXC IP address.
  • SSH_PATH - the full path to copy the exported content to. For me, this was /srv/http/blog/deployed/, meaning the root to serve in my reverse proxy is /srv/http/blog/deployed/.

There was also one secret:

  • SSH_PRIVATE_KEY - the raw private key (which was copied earlier).

Configuring the reverse proxy

I simply just dropped this in my Caddyfile:

nexy.blog {
    root * /srv/http/blog/deployed
    encode zstd gzip
    file_server
}

This tiny snippet set up automatic TLS on that domain, set the file server root to the configured /srv/http/blog/deployed/ directory, enabled on-the-fly zstd compression (with a fallback to gzip) to reduce bandwidth sent over the wire, and finally, the static file server.

One caddy reload later, and I was ready to serve my blog myself.

Pushing my changes

At this point, the workflow was already installed, and now, I've set it up. All that was left to do was push some changes!

And sure enough, I pushed an empty commit, watch the pre-existing build job whir away, and then watched as the deploy job started picking up steam, and then 20 seconds later, I refreshed nexy.blog, and there it was - my freshly produced blog!

Also, I now have a proofs page. You can verify some of my online identities via one of the following URLs: