For various reasons, I recently decided to start running a homelab again.\nIn this post I'll give a broad overview of the architecture I settled on,\nand why I made specific choices.\nI'll try to keep things relatively high level here,\nand we'll dive into the implementation details in future posts.
\nLet's start from the beginning with what I'm trying to achieve.\nI want to host multiple services, some of them internet-accessible,\nfrom a box on my desk running FreeBSD.\nSpecifically FreeBSD, because I find it has an uncommon trio of properties:
\nIt's surprisingly rare to find all 3 in a system.\nThe last 2 in particular are often mutually exclusive.
\nI already have the hardware (an old NUC, which I've named bsdcube even though it's not a cube),\nand my home internet is rock solid, but I don't have a static IP.\nEven if I did, I'm not sure I want to run some services on a residential IP,\nor to invite attacks on my home network.
The architecture that I chose keeps bsdcube on my home network behind the NAT firewall.\nTo make the server accessible from the internet,\nI use a WireGuard tunnel to a remote host that is internet-accessible.\nPF rules on the accessible server route certain traffic over the tunnel and block the rest.
+----------+ +---------+ +---------+\n | Internet |------>| proxbox |<---WireGuard-Tunnel---->| bsdcube |\n +----------+ +---------+ +---------+\nproxboxFor my internet-accessible box, which I dubbed proxbox,\nI wanted a reliable host that supported a BSD.\nI was initially planning on getting a FreeBSD box for the job.\nI even had a box set up, as described in my post on the subject.\n(I guess I'll need to write a follow-up now!)
I was somewhat discouraged by the poor support of FreeBSD on most cloud providers.\nThere is ironically a split where a lot of the mega-hosts like AWS have tier 1 support,\nbut there's nothing in the middle.\nMany of the midrange hosts which had good FreeBSD VM support now require a tedious manual install process.\nVultr seemed to be the only host that had at least some level of support,\nbut they were not my first choice.
\nThen my friend Andrew mentioned OpenBSD Amsterdam.\nIt was love at first sight.\nI had, somehow (despite being a FreeBSD using since around 2002) never actually used OpenBSD.\nBut I loved their pitch: a community-focused host that donates a large part of every purchase\nto the OpenBSD foundation.
\nThe onboarding was fast and easy.\nI filled out the form, and within 2 hours I had an email with connection instructions!\nI hadn't even been given an invoice yet!\nReally fantastic service.
\nproxbox setup using the OpenBSD base systemNow let's talk about the guts of proxbox.\nSurprisingly, almost everything running on this box is part of the OpenBSD base system!
+-------+ \n /--\\ +-----HTTP----->| httpd | \n / \\ ----+ | | \n \\ PF / +-------+ +---------+ \n \\__/ ----+ +--------+ +---blog--->| Varnish |--+ +---------------+\n | | relayd |--+ | | +-->| WG -> bsdcube |\n +-----HTTPS---->| |--+ +---------+ +-->| |\n +--------+ | | +---------------+\n +-------matrix-----------+ \nIt all starts with PF, the OpenBSD Packet Filter.\nIn addition to rejecting invalid / unwanted traffic,\nit's responsible for routing the legit packets to the right service.
\nhttpd(8) and relayd(8)An HTTP server sits on the box for handling the ACME challenges\n(e.g. from Let's Encrypt) to get a TLS cert.\nOpenBSD has httpd(8),\nnot to be confused with the Apache httpd.\nIt can serve static files and FastCGI, but it's not a reverse proxy.
Normally I'd reach for something like nginx, HAProxy, or Caddy,\nbut OpenBSD also has an excellent (load balancing) reverse proxy in the base system already:\nrelayd(8).
relayd in my setup is responsible for 2 things:\nrouting to the correct backend (e.g. based on SNI),\nand TLS termination.\nI'll write a longer post about this soon,\nincluding how to configure acme-client(1)to get certs automatically,\nand configuring relayd to terminate TLS for multiple domains.
Since my blog is just a static site,\nit would be quite reasonable to host it with httpd on proxbox.\nThis would result in faster response times,\nbut I decided to host it on bsdcube to make the setup simpler.\nThe clear separation of concerns, with bsdcube being the final source of truth\nfor all data and services makes it easier for me to maintain.\nI'll never have to touch proxbox outside of routine system maintenance or adding a new service port.
That said, I'm not going to just blindly forward every single request over to my home network.\nI've used Varnish Cache (now rebranded as Vinyl)\nfor over a decade in various professional contexts.\nIt does an excellent job at absorbing traffic spikes,\nand smoothing over issues when the backend goes flaky.\nSince I'm tunneling everything over the internet,\nhalfway around the world,\nthis seems like table stakes.
\n(Fun fact: the author of Varnish is also the author of FreeBSD's jails;\nhe's a legend in the FreeBSD community, and should be more widely known.)
\nWhatever route a legitimate packet takes,\nif it needs to talk to a backend service,\nthis happens over a WireGuard tunnel.
\nI could have done this with something like Tailscale or Cloudflare Warp,\nbut I wanted something that wasn't reliant on a "free" external service.\nSuch services are rarely free forever.\nRolling your own tunnel with a proven technology is surprisingly easy,\nand my intentional choice of a somewhat offbeat rest of the stack made the decision a no-brainer.\nBoth FreeBSD and OpenBSD have WireGuard support built-in at the kernel level already.
\nPF rules on both sides of the tunnel ensure that only certain traffic passes through.\nI will probably write a post on this later as well as there's quite a lot here.
\nbsdcubeLet's look at the other side of the tunnel now.\nbsdcube sits on my home network behind a NAT firewall.\nIt only accepts traffic over the tunnel for specific service ports.\nAnd those ports are mapped to jailed services with a limited view of the system.\nIn case you're not familiar with FreeBSD, jails are a FreeBSD-native containerization technology\nwhich has been around for over 25 years (that's a long time before Docker popularized the term).\nWhile they have many differences with Linux containers,\nthat's a reasonable mental model for understanding the rest of this post.
The base system has jail(8) as a basic management facility.\nThis does technically have everything you need,\nbut it requires enough steps that,\njust as in the Linux container world,\nmost people rely on higher level tools for management and orchestration.
There are over a dozen such tools for FreeBSD,\neach with a different take on things.\nThat's both cool (healthy ecosystem!) and bewildering for newcomers to the space ;)
\nI ended up settling on AppJail for bsdcube,\nat least for now,\nsince it has extensive documentation,\nsupport for features I'm curious about later (like Linux jails),\nand a declarative, composable configuration format.
Makejail filesHere's a sample of a Makejail file.\nAs you might guess, it's a set of instructions for building a jail.\nIt looks a bit like a Dockerfile.
# blog/Makejail\n\n# Include directives let you separate out sections of your config across files\n# and reuse common sections.\nINCLUDE options/options.makejail\nINCLUDE options/network.makejail\n\n# Install nginx from the FreeBSD package repository\nPKG nginx\n\n# A sysrc directive.\n# This is the equivalent of enabling a service on a Linux box with systemd\nSYSRC nginx_enable=YES\n\n# Copies nginx.conf from the current directory into the specified path in the jail.\nCOPY nginx.conf /usr/local/etc/nginx/nginx.conf\n\n# Install Marmite, the static site generator I use for my blog.\n# Despite being relatively obscure, there's also an up-to-date package for this!\nPKG marmite\n\nINCLUDE buildsite.makejail\n\nSERVICE nginx start\n"Scripted" setups like this can get pretty unwieldy if you're not careful.\nAnd for me, one of the worst parts of CLI tools is remembering the half dozen switches\nto get a particular task done.
\nTo solve this, I structured my jail deployments as a git repository\nwhere each subdirectory is a self-contained service.\nIn my setup, there are no dependencies between services.\nI'm deploying full applications/services in isolated jails,\neven when I could theoretically have shared services (e.g., a Postgres database).\nAs such, it's easy to add a justfile to each directory\nwith deployment recipes so I don't have to remember them.
The usual invocation is something like just freshjail to (re)create a jail.\nAnd for jails like my blog, where the application has no need to restart\nto get an update, I have recipe(s) that perform the necessary updates against the running jail.
That was a lot... and there's still a lot I didn't cover!\nLike tips on the AppJail host setup, PF, handling multiple hosts with SNI via relayd, and a lot more.\nSo stay tuned for the next post,\nand give a shout on Mastodon if you have any feedback.
In the meantime, I've uploaded the AppJail part of my setup\nto a public repo on Codeberg.\nHopefully they are useful for others!
\n", "summary": "", "date_published": "2026-04-11T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "homelab", "FreeBSD", "OpenBSD", "networking", "jails", "containerization" ], "language": "en" } ] }