QUICK NOTE: Hardware is expensive nowadays

The AI boom came with very unfortunate side effects for the hobbiests that enjoy buying hardware. Hardware is very expensive nowadays, especially components such as RAM, GPUs, and Storage. It used to be very easy to buy these parts on the used market, or find a well-priced, pre-built server, used or new depending on your budget. That is no longer the case at the time of writing this.

Since I’ve been playing around with my homelab for a few years now, I managed to miss the AI boom hardware price apocalypse. For those just getting started and interested in buying hardware to have at home, I recommend repurposing anything extra you have now or can get your hands on, and keep a close lookout for deals. Kind of obvious advice, but hopefully this helps someone.

You don’t need to start out with a full kitted setup. An extra laptop you have lying around will do. I personally use mini-pc’s for my setup and they are perfect for my needs and for saving some space at home.

What is Proxmox? A brief intro

Proxmox is a company that makes fantastic infrastructure tools, such as Proxmox Virtual Environment (PVE) and Proxmox Backup Server (PBS).

When talking about Proxmox, most people are referring to the Proxmox Virtual Environment. PVE is a level 1 hypervisor, meaning it is installed directly onto the hardware rather than it being software you install separately on your desktop, like VirtualBox; PVE is essentially the OS. Hypervisors are used to virtualize hardware so you can split the resources on the physical hardware into smaller sized virtual machines. PVE has some subscriptions and support you can pay for, but a big portion of the features are fully unlocked and freely available when you install PVE.

Proxmox also makes Proxmox Backup Server as a separate appliance from PVE. PBS integrates seamlessly with PVE once it’s installed and up and running. It is also freely available for you to download and use. PBS is also a feature rich and easy to use appliance that rivals expensive Enterprise products.

Proxmox has always been well known in the virtualization community for their high quality products and features, and a staple of the homelab and self-hosted community. Recently, it has had a surge of popularity and adoption after the great VMWare exodus due to the Broadcom acquisition. It has always been a great product to pickup and learn, and it has always been a great choice for Small Businesses and Enterprise, but VMWare has historically overshadowed everything in this category until recently. VMWare was a tough product to beat, but nowadays it’s a tough product to justify the cost for.

Installing Proxmox at home, or even in a decent sized VM is a great way to pick up and learn a solid virtualization platform. Try to build out your environment like it was a Small Business, and treat it as such, and you’ll learn a lot about keeping infrastructure running and keeping the lights on.

Overview of my Proxmox Architecture

Since Proxmox is an on-prem tool, it’s mostly installed on hardware.

Hardware specs

My Proxmox hardware looks like:

  • 3 Minisforum MS-01
  • 2x2TB SSD, 1x1TB SSD on each node
  • 64GB of RAM on each node

Proxmox and Cluster config overview

  • Proxmox setup with Ceph during install
  • NICs dedicated for Management and VMs, Corosync, and Ceph Public/Private traffic
    • this also means separate networks for each
  • Cluster initialized on one node in the Datacenter settings
  • other nodes joined to the Cluster from the Datacenter settings using the join info
  • Ceph configured for shared storage amongst the nodes

What’s next?

This was a very surface-level overview of my Proxmox setup at home. Trying to add everything about my setup in one article would be a lot for one page to handle so I hope to add more about my setup bit by bit with more details about specific topics, share how I did it and what configs worked, and what I would do different today if I were to build it again from scratch. The way I handled my Ceph networking at the time I set it up is a great example of something I would do differently today and would be pretty troublesome to reconfigure now.