tl;dr
I have a basic home server with one NVMe drive, multiple SSDs, and multiple (SATA) HDDs, plus multiple USB3 ports and an SD card reader.
What’s the best boot / storage configuration for good performance with a basic degree of resilience? Thank you.
More
First, a disclaimer:
In an ideal world I’d have multiple hosts, for redundancy and failover. I’d also have separate storage: a well-spec’d FreeNAS box with a decent SAS card and a healthy chunk of ECC memory, serving shares via NFS. But at the moment I don’t have those things; I’m trying to do the best I can with what I’ve got.
So:
BOOT
I’ve read some documentation and blog posts. I’ve seen recommendations to boot from SD card, but I’m a bit wary of this: my experience with SBCs (e.g. RPi) is that after a while - when the card starts to run out of write cycles - they can fail in interesting and unpredictable ways. I have experience booting OSes from USB as well (see FreeNAS, above); do you think this would be a better option?
STORAGE
In terms of “speed”: NVMe > SSD > HDD
…So I’m assuming:
SSDs (mirrored): VMs
HDDs (mirrored): other persistent data
NVMe: cache and ephemeral data (etc.)
Does that make sense? Is there a better way?
Thank you for reading Apologies for a long post.
P.S. Out-of-scope-but-still-relevant: the server has 32G of (non-ECC) RAM, and 5 x 1G Intel ethernet. My goal is to host no more than three “permanent” VMs (e.g. LAMP/LEMP), plus occasional others for experiments I can’t do - or don’t want to do - in a container / virtual environment. Thanks again.