Best use of two 2TB NVMe drives
I'm running a homelab, and want to make the best use of my hardware. Nothing needs to be production secure, I just want to make sure I don't lose archived data. The most important stuff is properly backed up, but obviously I'd rather not lose the unimportant stuff either.
I've got two 2TB NVMe drives available for my 8x16TB raidz2 server and I'm honestly not sure how best to use them. They're WD Red drives, so a great 2500 TBW of write endurance. The read/write speed isn't as good as some others (a measly 3GB/s), but I got them at an incredibly good price so I can't complain. I'd like to be able to use the performance they have, and while read performance is probably more important than write performance, if there's a simple way to get the best of both worlds then it can't hurt.
Initially I was just going to mirror them and use them for apps or whatever, essentially just acting as fast local storage. I can't help but feel like that's a waste of them though, so I've been trying to figure out a better use.
The obvious candidates are either a Special Metadata Device, L2ARC, or SLOG. I've done a decent enough amount of reading to know roughly what these do and how they work, but I can't figure out which is the best, safest, and most impactful option.
A Special Metadata Device seems like a good choice, as it would make accessing the array much snappier so it would probably be felt. At the same time it only really impacts small file reads, and from my understanding doesn't do anything for transfer speeds in either direction. It makes me a little anxious though, since if somehow both drives failed I could end up losing my array. I don't know how realistic a worry this actually is compared to losing a regular hard drive and I'm assuming that if there was a catastrophic failure and both drives ended up read only I could probably still recover the data, but it definitely makes me anxious. From what I can tell write endurance isn't an issue when using this, since it's essentially just a table. It also feels like it might be a waste of space, since the drives are pretty big and the metadata will probably never end up taking more than 500GB.
Similar to this is a Dedup vdev, which seems like it could be useful but I assume everyone will recommend against. This has the same concern of losing it taking down the whole pool, but from what I can tell it also requires a much higher write endurance than a special metadata device would due to the dedup process. This feels like a good use for two mirrored 280gb optane modules, but honestly it doesn't feel like a good fit for my hardware. Theoretically it could improve write performance while saving a ton of space for something like Steam Libraries for multiple users, but honestly improving read for that is much more important than improving write and realistically it's only going to be me. I am a big fan of dedup generally, but honestly I feel like having a vdev for it isn't all that necessary even if I was using it and I'd probably be better served by a L2ARC for the metadata.
L2ARC feels like it would be a good use, though I've seen plenty of people saying "Don't use L2ARC" so I've honestly got no idea what to think. My understanding of it is that it's a second level read cache for frequently read files that are too big for RAM/spillover for when RAM is full, and it's "safe" in that a drive failure would lose no data. Since this is volatile and safe I'm guessing I could conceivably stripe the drives instead for a ton of space and performance, which should give me blisteringly fast access to frequently used data beyond what RAM can hold. I can see this being beneficial for video editing and an iSCSI Steam library, which would be pretty great uses of the server. However, I've also read that L2ARC is one of the most write intensive choices and as such write endurance becomes a relevant factor with it.
The one I'm the least confident with is a SLOG. My understanding is that it's essentially a write cache, and that running one could improve the performance in terms of latency of small synchronous writes. I see a lot of people scaremongering about drive failure for a SLOG, but my understanding is that would probably be... fine? As far as I can tell if theoretically I was running a big production database and everything died in the middle of a load of data writes I could lose some data, but for a little home server it probably won't matter. At the same time, it also probably won't really improve performance to any notable degree. I've also read that TrueNAS only uses 16GiB for the SLOG and that due to it being a "write cache" write endurance is the most important statistic, so this feels like a bad fit for 2TB NVMe drives and I'm almost certainly better just using a small 16GB Optane module for this instead of a regular drive.
Assuming my understanding of all of these things is correct, what would you recommend I do for overall performance? A striped L2ARC for my main pool, a mirrored separate pool specifically for data I want to access quickly, a single drive L2ARC with a small Optane module for a SLOG, a mirror for a Special Metadata Device, or something else entirely?
I'd really appreciate any advice, thanks!