I was asked a few times from friends and William Lam from virtuallyghetto collects how HomeLabs from the community looks like, so I thought I’m just going ahead writing a "short" page about my current setup. Latest update was 2022-01-10.
To be more exact, my HomeLab consists out of multiple HomeLabs – three, to be precise. Starting with the most boring one…
HomeLab #1: Germany, Nuernberg.
This is a Dell PowerEdge R730 hosted in Germany, Nuernberg in a datacenter at Hetzner Online GmbH. As this is a rented server, no picture is available – you could search for the server model in a search engine of your choice… but it looks like how a server looks like.
Server Specs:
- CPU: Intel Xeon E5-2620 v4
- RAM: 64 GB DDR4 RAM
- Storage with Hardware-RAID
- 4x 2 TB Enterprise HDD (RAID10)
- 2x 250 GB Datacenter SSD (RAID1)
- Running VMware vSphere ESXi 7.0b
Purpose: As the server runs in a datacenter, this server mostly only runs services which should be publicly accessible – like my mailserver, webserver (where also this website runs on), Nextcloud instance, etc.
HomeLab #2: Ireland, Cork. (Home)
Due to lack of space and possibilities wiring every single corner of the appartment (hi landlord!), the HomeLab is placed on my desk just next to me. Therefore it really needs to be as quiet as possible – otherwise I might drive crazy at some point.
A more detailed list…
Probably the most interesting bits:
-
Compute Node 1
- Model: Intel NUC BXNUC10I7FNH2
- CPU: Intel Core i7-10710U
- RAM: 2x Crucial CT2K32G4SFD8266 32 GB DDR4
- Disks
- Storage 1: Samsung 970 EVO Plus 250 GB NVMe M.2 Internal SSD
- Storage 2: Samsung 860 EVO 2 TB SATA 2.5" Internal SSD
- Boot medium: SanDisk Cruzer Fit 16GB USB 2.0 Flash Drive
- Network
- 1x 1 GBe built-in, Management
- 2x 1 GBe USB-NICs StarTech USB 3.0 to Dual Port Gigabit Ethernet Adapter via USB-C (using USB-NIC-Driver-Fling). Using for iSCSI and vMotion.
- 1x 2.5 GBe USB-NIC Club 3D USB 3.2 Gen1 Adapter Type A to 2.5 Gigabit Ethernet for vSAN traffic.
- Running VMware vSphere ESXi 7.0 U1 (7.0 works out-of-the-box, also check out HomeLab 7.0 considerations by William here)
- Purpose: Running all kind of workload. Running vSAN.
-
Compute Node 2
- Model: Intel NUC BXNUC10I7FNH2
- CPU: Intel Core i7-10710U
- RAM: 1x Crucial 16 GB DDR4 + 1x Crucial 8 GB DDR4
- Disks
- Storage 1: Samsung 970 EVO Plus 250 GB NVMe M.2 Internal SSD
- Storage 2: Samsung 860 EVO 2 TB SATA 2.5" Internal SSD
- Boot medium: SanDisk Cruzer Fit 32GB USB 2.0 Flash Drive
- Network
- 1x 1 GBe built-in, Management
- 2x 1 GBe USB-NICs StarTech USB 3.0 to Dual Port Gigabit Ethernet Adapter (using USB-NIC-Driver-Fling). Using for iSCSI and vMotion.
- 1x 2.5 GBe USB-NIC Club 3D USB 3.2 Gen1 Adapter Type A to 2.5 Gigabit Ethernet for vSAN traffic.
- Running VMware vSphere ESXi 7.0 U1
- Purpose: Running all kind of workload. Running vSAN.
-
Compute Node 3
- Model: Intel NUC NUC7i5BNH
- CPU: Intel Core i5-7260U
- RAM: 2x 16 GB
- Disks
- Storage 1: Samsung 860 EVO 250 GB M.2
- Storage 2: Samsung 870 EVO 1 TB SATA
- Boot medium: SanDisk Cruzer Fit 32GB USB 2.0 Flash Drive
- Network
- 1x 1 GBe built-in, Management (+ vSAN Witness Traffic)
- 2x 1 GBe USB-NICs Anker USB 3.0 Portable Gigabit Ethernet Adapter (using USB-NIC-Driver-Fling). Using for iSCSI and vMotion.
- Running VMware vSphere ESXi 7.0 U1 (7.0 works out-of-the-box, also check out HomeLab 7.0 considerations by William here)
- Purpose: vSAN Witness Node, Backup Proxy (Management Cluster)
-
Compute Node 4
- Model: Intel NUC NUC8i5BEH
- CPU: Intel Core i5-8259U
- RAM: 2x 16 GB
- Disks
- Storage 1: Samsung 860 EVO 250 GB M.2
- Storage 2: Samsung 870 EVO 1 TB SATA
- Boot medium: SanDisk Cruzer Fit 32GB USB 2.0 Flash Drive
- Network
- 1x 1 GBe built-in, Management (+ vSAN Witness Traffic)
- 2x 1 GBe USB-NICs Anker USB 3.0 Portable Gigabit Ethernet Adapter (using USB-NIC-Driver-Fling). Using for iSCSI and vMotion.
- Running VMware vSphere ESXi 7.0 U1 (7.0 works out-of-the-box, also check out HomeLab 7.0 considerations by William here)
- Purpose: vSAN Witness Node, Backup Proxy (Management Cluster)
-
Synology NAS
- Model: Synology DS918+ 4 Bay Desktop NAS Enclosure
- Additional memory: Synology D3NS1866L-4G 4 GB DDR3 RAM Module (upgraded to maximum of 8 GB in total)
- Storage: 4x Western Digital WD40EFRX 4 TB in SHR-RAID (similar to RAID5, 12 TB useable space)
- Read cache: 1x Samsung 860 EVO 250 GB SATA M.2
- Purpose: Datagrave, backups, Plex Media Server, and iSCSI target.
-
Firewall
- Model: NRG Systems IPU672
- Actually a re-branded Qotom Mini PC.
- Running OPNsense.
- CPU: Intel Core i5-7200U Kaby Lake-U
- RAM: 8 GB DDR4 Crucial RAM
- Disk: 250 GB Samsung SSD SATA
- Purpose: Doing firewall stuff.
Less interesting ones:
- Synology DS1513+
- Purpose: No idea. Owned by my flat mate. Attached to my UPS.
- Philips Hue Bridge
- RIPE Atlas Probe
- Raspberry Pi 4 (4 GB RAM)
- Raspberry Pi 3B+
Network-wise:
- NETGEAR GS116E 16-Port Gigabit Ethernet Smart Managed Plus Network Switch
- NETGEAR GS108Tv2 8-Port Gigabit Smart Managed Pro Switch
- WLAN: Unifi FlexHD
If someone cares, some pictures:
HomeLab #3: Austria, Lower Austria.
This might be the longest bit. It started with a HPE MicroServer Gen8… and somehow… it escalated quickly. Make a new coffee before you continue reading, might take a while!
So… where do we start…
-
1st/2nd Compute Nodes, each one with following specs:
- Model: HPE ProLiant DL380 Gen9
- CPU: 2x Intel Xeon E5-2667 v4 @ 3.20GHz (32 threads)
- RAM: 320 GB DDR4 RAM (10x 32 GB)
- Boot: Samsung 980 500 GB NVMe M.2 via NVMe PCIe x16 Adapter
- Disks (via HPE Smart Array P840ar)
- 10x HPE 1.2 TB SATA HDD
- 2x Micron 5300 Max 960GB 2.5 (vSAN Capacity Disks)
- Dual PSU to two independent UPS, with TPM 2.0
- NICs: HP Ethernet 10Gb 2-port 560FLR-SFP+ Adapter
- Running VMware vSphere ESXi 7.0 U2
-
3rd Compute Nodes
- Model: HPE ProLiant DL380 Gen9
- CPU: 2x Intel Xeon E5-2673 v4 @ 2.30GHz (40 threads)
- RAM: 384 GB DDR4 RAM (12x 32 GB)
- Boot: Samsung 980 500 GB NVMe M.2 via NVMe PCIe x16 Adapter
- Disks (via HPE Smart Array P440ar)
- 5x HPE 1.2 TB SATA HDD (vSAN Capacity Disks)
- 1x Micron 5300 Max 960GB 2.5 (vSAN Cache Disks)
- Disks (via HPE Smart HBA H240)
- 5x HPE 1.2 TB SATA HDD (vSAN Capacity Disks)
- 1x Micron 5300 Max 960GB 2.5 (vSAN Cache Disks)
- Dual PSU to two independent UPS, with TPM 2.0
- NICs: HP Ethernet 10Gb 2-port 530 FLR-SFP+ Adapter
- Running VMware vSphere ESXi 7.0 U2
-
Storage
- Model: HPE ProLiant DL360p Gen8
- CPU: 2x Intel Xeon E5-2650L v2
- RAM: 96 GB DDR3 RAM
- Disks (via HPE Smart HBA H240)
- 2x Kingston SSDNow UV400/500 (Bootmedium, RAID1)
- 2x SAMSUNG SSD SM883 240GB (Cache 1 for RAID10)
- 2x SAMSUNG SSD SM883 240GB (Cache 2 for RAID6)
- 2x SAMSUNG SSD SM883 240GB (LOG for both RAIDs)
- Disks (via HPE Smart Array P421)
- 7x Western Digital Red 1TB (ZFS RAID10, 1x Hot-Spare)
- 5x Western Digital Red 10TB (RAID6/RAIDZ2)
- Dual PSU to two independent UPS
- NICs: HP Ethernet 10Gb 2-port 530 FLR-SFP+ Adapter
- Running FreeNAS/TrueNAS
- Purpose: iSCSI/NFS Storage for ESXi Hosts, mainly for backups
-
Total of 3x UPS
- 2x APC Smart-UPS SMT – SMT1500I 1500VA with Network Management Module
- Each UPS is connected to each server: Each server has Dual-PSU, connected to two differenet UPS for increased redundancy. Also the clustered switches are each connected to a different UPS.
- Purpose: Powering all servers, storage and redundant switches.
- 1x APC Back-UPS PRO 900VA
- Purpose: Powering less essential stuff like modem, Raspberry Pi, management switch, etc.
- 2x APC Smart-UPS SMT – SMT1500I 1500VA with Network Management Module
-
Raspberry Pi 3B + GPS HAT
- Purpose: NTP Timeserver (Stratum 1) via chronyd. Pulls current time from GPS satellites, using PPS (Pulse-per-second) for most accurate timing.
Network-wise:
Beside the hardware-wise isolation (Core/SAN), there are also about 15 VLANs across all switches to isolate various workloads from each other.
-
Core Switch
- Model: 2x HPE Aruba 2930F-24G-4SFP Switch (JL259A)
- Those two switches are running in a VSF stack, so they are clustered for increased reduncancy.
- Purpose: Main switch handling all workload like traffic from/to virtual machines, Internet traffic and OSPF/BGP to/from firewall. Uplink over fiber-optic cable to upper floor. Servers are wired to both switches for increased redundancy. LACP in use with Virtual Distributed Switches on the ESXi hosts.
-
SAN Switch
- Model: MikroTik CCR2004-1G-12S+2XS (12x 10 GBe, SFP+ + 2x 25 GBe)
- Purpose: Mainly handling storage traffic for iSCSI, NSX-T, vSAN and vMotion.
-
Management Switch
- Model: HPE Aruba 2920-24G-4SFP Switch (J9726A)
- Purpose: Handling management traffic like iLO to each HPE server, internet traffic from/to modem in separate VLAN, access points, Raspberry Pi GPS time server, etc. There is a uplink of 2x 1 GBe fibre-optic cable to the Core Switch.
-
Firewall: 2x WatchGuard XTM 525 Appliances (unfortunately EOL)
-
Access Switch
- Model: NETGEAR GS110TP 8 Port PoE Gigabit Smart Managed Pro Switch
- Purpose: This switch is for connecting upper floor (where fiber-optic cable from basement is connected to) and below two switches to handle e.g. client traffic.
-
Client Switches
- Model: 2x NETGEAR GS108Tv2 8-Port Gigabit Smart Managed Pro Switch
- Purpose: Handles media area (TV/Xbox) and office (1x PC and 1x network printer)
Running following VMware products:
- VMware vSphere vCenter
- VMware vSAN
- VMware NSX-T
- VMware NSX Advanced Load Balancer
- VMware Horizon
- VMware Unified Access Gateway
- VMware AppVolumes
- VMware vRealize Log Insight
- VMware vRealize Operations Manager
- VMware vCloud Director
- Carbon Black
And in case you’re wondering: Yes, that are 2x Philips Hue Light Strips. Because of reasons.
Also, here are some pictures: