Join Vultr Community
Points, levels & badges
Enter challenges & submit builds
Register to events & meetups
Get your work featured
Moving a multi-tenant compliance platform to Vultr bare metal with Incus
Eric Adams
NoviceHi, I’m Eric, based in Louisiana. My background is in occupational safety and compliance, including hazmat transportation. I also build and operate compliance platforms for DOT and OSHA consulting.
The platform is a white-label suite for driver qualification, hot work and confined space permits, OSHA certification tracking, policy management, and hiring workflows. Since regulators, insurers, or attorneys may review its records, I prioritize traceability, least-privilege access, SSO across the apps, and a small dependency footprint.
I’m consolidating multiple cloud compute instances onto two Vultr bare-metal hosts. Each host has:
- CPU: 1 × AMD EPYC 4345P
- Cores: 8 cores / 16 threads at 3.8 GHz
- Cache and power: 32 MB L3 cache, 65 W TDP
- Memory: 128 GB DDR5
- Disks: 2 × 1,945 GB NVMe in RAID 1
- Bandwidth: 10 TB/month
The hosts will run Ubuntu 26.04 and Incus. Each existing server will become a guest with the same name and contents. Most will be system containers. The guest handling uploaded documents and headless Chrome will be a VM because it runs the least-trusted workloads.
The identity provider will stay on a separate small server. Postgres is on Neon, files are on R2, and backups are on B2. There will be no database or Docker on the hosts. CI and application backups run through GitHub Actions with alerting.
I’d be interested in hearing how others approach:
- Storage pools on mirrored disks: ZFS, btrfs, or LVM? How do you handle guest snapshots and off-host exports?
- Resource limits and isolation for a VM and containers handling workloads with different trust levels?
- How do you time and verify a full rebuild of a multi-host setup? What did your first drill uncover?
- What’s your approach to logging and evidence retention for audit-sensitive workloads on bare metal?
Looking forward to the discussions.
RV
1 comment · incl. Vultr Team1 Comment
Most Helpful — awarded automatically on community upvotes
4
Commented: 6 days ago
Hey! Welcome to Vultr Community!
I am not very versed with Incus but I personally use Proxmox with ZFS to manage my nodes, along with Proxmox Backup Server (PBS) to manage the ZFS snapshots and deduplicated backups. So far it has been running flawlessly. Maybe I can give you some insight on how I do things using this setup :)
- Storage pools on mirrored disks: ZFS, btrfs, or LVM? How do you handle guest snapshots and off-host exports?
PBS allows you to add different datastores (including an S3 compatible datastore) or NFS/SMB which you can use to store your LXC/VM backups off-premise.
I was also looking into some storage vendors that support ZFS-send natively so that my backup workflow becomes easier and faster (as ZFS snapshots take insanely long time to copy using rsync) but I couldn't find any reliable/production-ready or battle tested solutions (though, https://rsync.net/ seemed promising but they support ZFS-send only if you subscribe to their 10TB plan)
- How do you time and verify a full rebuild of a multi-host setup? What did your first drill uncover?
I never actually looked into this since I use my cluster for personal use as a homelab. But I believe there is a way to have an automated install for Proxmox atleast and believe there might be something similar for IncusOS. Otherwise maybe looking into IaC tools like terrraform or Ansible (along with PXE boot?) since you prefer using Incus with an Ubuntu host.
- What’s your approach to logging and evidence retention for audit-sensitive workloads on bare metal?
Again, not my usecase so unfortunately cant really help you here :( But maybe SaaS tools like Vanta or Sprinto can help there. I knnow they have integrations for cloud infrastructures, not sure how good their support is for on-prem solutions.
4