Article automatically read by Amazon's Text-To-Speech Engine: |
I was asked a few times from friends and William Lam from virtuallyghetto collects how HomeLabs from the community looks like, so I thought I’m just going ahead writing a "short" page about my current setup. Latest update was 2020-11-03.
To be more exact, my HomeLab consists out of multiple HomeLabs – three, to be precise. Starting with the most boring one…
HomeLab #1: Germany, Nuernberg.
This is a Dell PowerEdge R730 hosted in Germany, Nuernberg in a datacenter at Hetzner Online GmbH. As this is a rented server, no picture is available – you could search for the server model in a search engine of your choice… but it looks like how a server looks like.
Server Specs:
- CPU: Intel Xeon E5-2620 v4
- RAM: 64 GB DDR4 RAM
- Storage with Hardware-RAID
- 4x 2 TB Enterprise HDD (RAID10)
- 2x 250 GB Datacenter SSD (RAID1)
- Running VMware vSphere ESXi 7.0b
Purpose: As the server runs in a datacenter, this server mostly only runs services which should be publicly accessible – like my mailserver, webserver (where also this website runs on), Nextcloud instance, etc.
HomeLab #2: Ireland, Cork. (Home)
Due to lack of space and possibilities wiring every single corner of the appartment (hi landlord!), the HomeLab is placed on my desk just next to me. Therefore it really needs to be as quiet as possible – otherwise I might drive crazy at some point.
A more detailed list…
Probably the most interesting bits:
-
Compute Node 1
- Model: Intel NUC BXNUC10I7FNH2
- CPU: Intel Core i7-10710U
- RAM: 2x Crucial CT2K32G4SFD8266 32 GB DDR4
- Disks
- Storage 1: Samsung 970 EVO Plus 250 GB NVMe M.2 Internal SSD
- Storage 2: Samsung 860 EVO 2 TB SATA 2.5" Internal SSD
- Boot medium: SanDisk Cruzer Fit 16GB USB 2.0 Flash Drive
- Network
- 1x 1 GBe built-in, Management
- 2x 1 GBe USB-NICs StarTech USB 3.0 to Dual Port Gigabit Ethernet Adapter via USB-C (using USB-NIC-Driver-Fling). Using for iSCSI and vMotion.
- 1x 2.5 GBe USB-NIC Club 3D USB 3.2 Gen1 Adapter Type A to 2.5 Gigabit Ethernet for vSAN traffic.
- Running VMware vSphere ESXi 7.0 U1 (7.0 works out-of-the-box, also check out HomeLab 7.0 considerations by William here)
- Purpose: Running all kind of workload. Running vSAN.
-
Compute Node 2
- Model: Intel NUC BXNUC10I7FNH2
- CPU: Intel Core i7-10710U
- RAM: 1x Crucial 16 GB DDR4 + 1x Crucial 8 GB DDR4
- Disks
- Storage 1: Samsung 970 EVO Plus 250 GB NVMe M.2 Internal SSD
- Storage 2: Samsung 860 EVO 2 TB SATA 2.5" Internal SSD
- Boot medium: SanDisk Cruzer Fit 32GB USB 2.0 Flash Drive
- Network
- 1x 1 GBe built-in, Management
- 2x 1 GBe USB-NICs StarTech USB 3.0 to Dual Port Gigabit Ethernet Adapter (using USB-NIC-Driver-Fling). Using for iSCSI and vMotion.
- 1x 2.5 GBe USB-NIC Club 3D USB 3.2 Gen1 Adapter Type A to 2.5 Gigabit Ethernet for vSAN traffic.
- Running VMware vSphere ESXi 7.0 U1
- Purpose: Running all kind of workload. Running vSAN.
-
Compute Node 3
- Model: Intel NUC NUC7i5BNH
- CPU: Intel Core i5-7260U
- RAM: 2x Samsung 32 GB DDR4 (Yes, 64 GB works – see here)
- Disks
- Storage 1: Samsung 860 EVO 250 GB SSD
- Boot medium: SanDisk Cruzer Fit 32GB USB 2.0 Flash Drive
- Network
- 1x 1 GBe built-in, Management (+ vSAN WItness Traffic)
- 2x 1 GBe USB-NICs Anker USB 3.0 Portable Gigabit Ethernet Adapter (using USB-NIC-Driver-Fling). Using for iSCSI and vMotion.
- Running VMware vSphere ESXi 7.0 U1 (7.0 works out-of-the-box, also check out HomeLab 7.0 considerations by William here)
- Purpose: vSAN Witness Node, Backup Proxy
-
Synology NAS
- Model: Synology DS918+ 4 Bay Desktop NAS Enclosure
- Additional memory: Synology D3NS1866L-4G 4 GB DDR3 RAM Module (upgraded to maximum of 8 GB in total)
- Storage: 4x Western Digital WD40EFRX 4 TB in SHR-RAID (similar to RAID5, 12 TB useable space)
- Read cache: 1x Samsung 860 EVO 250 GB SATA M.2
- Purpose: Datagrave, backups, Plex Media Server, and iSCSI target.
-
Firewall
- Model: NRG Systems IPU672
- Actually a re-branded Qotom Mini PC.
- Running OPNsense.
- CPU: Intel Core i5-7200U Kaby Lake-U
- RAM: 8 GB DDR4 Crucial RAM
- Disk: 250 GB Samsung SSD SATA
- Purpose: Doing firewall stuff.
Less interesting ones:
- Synology DS1513+
- Purpose: No idea. Owned by my flat mate. Attached to my UPS.
- Philips Hue Bridge
- RIPE Atlas Probe
- Raspberry Pi 4 (4 GB RAM)
- Raspberry Pi 3B+
Network-wise:
- NETGEAR GS116E 16-Port Gigabit Ethernet Smart Managed Plus Network Switch
- NETGEAR GS108Tv2 8-Port Gigabit Smart Managed Pro Switch
- WLAN: Unifi AP-AC-LR
If someone cares, some pictures:
HomeLab #3: Austria, Lower Austria.
This might be the longest bit. It started with a HPE MicroServer Gen8… and somehow… it escalated quickly. Make a new coffee before you continue reading, might take a while!
So… where do we start…
-
3x Identical Compute Nodes, each one consists out of:
- Model: HPE ProLiant DL360p Gen8
- CPU: 2x Intel Xeon E5-2650L v2
- RAM: 96 GB DDR3 RAM
- Disks (via HPE Smart Array H240 HBA)
- 1x SAMSUNG SSD SM883 240GB (vSAN Cache Disk)
- 4x Samsung SSD 860PRO 512GB (vSAN Capacity Disks)
- Dual PSU to two independent UPS
- NICs: 2x HPE Quad Port 1 GBe, redundancy to two switches
- Running VMware vSphere ESXi 7.0 U1, booting from SD-Card
- Purpose: Doing all kind of workload. Using DRS/HA.
-
Storage
- Model: HPE ProLiant ML310e v2 Gen8
- CPU: Intel Xeon E3-1220 v3
- RAM: 32 GB DDR3
- Disks
- 2x SAMSUNG SSD SM883 240GB (Cache 1 for RAID10, RAID0)
- 2x SAMSUNG SSD 850 EVO 250GB (Cache 2 for RAID6, RAID0)
- 2x Kingston SSDNow UV400/500 (Bootmedium, RAID1)
- 7x Western Digital Red 1TB (ZFS RAID10, 1x Hot-Spare)
- 5x Western Digital Red 10TB (RAID6/RAIDZ2)
- Dual PSU to two independent UPS
- NICs: 2x HPE Quad Port 1 GBe, redundancy to two switches
- Running FreeNAS 11
- Purpose: iSCSI Storage for ESXi Hosts
-
Total of 3x UPS
- 2x APC Smart-UPS SMT – SMT1500I 1500VA with Network Management Module
- Each UPS is connected to each server: Each server has Dual-PSU, connected to two differenet UPS for increased redundancy. Also the clustered switches are each connected to a different UPS.
- Purpose: Powering all servers, storage and redundant switches.
- 1x APC Back-UPS PRO 900VA
- Purpose: Powering less essential stuff like modem, Raspberry Pi, management switch, etc.
- 2x APC Smart-UPS SMT – SMT1500I 1500VA with Network Management Module
-
Raspberry Pi 3B + GPS HAT
- Powered via PoE from Management Switch, to be able to restart if if required. Secondly it’s also powered through UPS that way.
- Purpose: NTP Timeserver via Chrony. Pulls current time from GPS satellites, using PPS (Pulse-per-second) for most accurate timing.
Network-wise:
Beside the hardware-wise isolation (Core/SAN), there are also about 15 VLANs across all switches to isolate various workloads from each other.
-
Core Switch
- Model: 2x HPE Aruba 2930F-24G-4SFP Switch (JL259A)
- Those two switches are running in a VSF stack, so they are clustered for increased reduncancy.
- Purpose: Main switch handling all workload like traffic from/to virtual machines, Internet traffic and OSPF/BGP to/from firewall. Uplink over fiber-optic cable to upper floor. Servers are wired to both switches for increased redundancy. LACP in use with Virtual Distributed Switches on the ESXi hosts.
-
SAN Switch
- Model: 2x HPE Aruba 2920-24G-4SFP Switch (J9726A)
- Those two switches are clustered using dedicated cluster modules on the back-side of the switches. The modules are connected via one DAC cable over 40 GBe.
- Purpose: Mainly handling storage traffic for iSCSI and vSAN, but also vMotion and internal NSX traffic are going through this switch. Servers are wired to both switches for increased redundancy.
-
Management Switch
- Model: HP Switch 3500yl 24G (J8692A)
- Purpose: Handling management traffic like iLO to each HPE server, internet traffic from/to modem in separate VLAN, access points, Raspberry Pi GPS time server, etc. There is a uplink of 2x 1 GBe fibre-optic cable to the Core Switch.
-
Firewall: 2x WatchGuard XTM 525 Appliances (unfortunately EOL)
-
Access Switch
- Model: NETGEAR GS110TP 8 Port PoE Gigabit Smart Managed Pro Switch
- Purpose: This switch is for connecting upper floor (where fiber-optic cable from basement is connected to) and below two switches to handle e.g. client traffic.
-
Client Switches
- Model: 2x NETGEAR GS108Tv2 8-Port Gigabit Smart Managed Pro Switch
- Purpose: Handles media area (TV/Xbox) and office (1x PC and 1x network printer)
On said HomeLab also runs:
- VMware vSphere vCenter
- VMware vSAN
- VMware NSX-T
- VMware NSX Intelligence
- VMware Horizon
- VMware AppVolumes
- VMware vRealize Log Insight
- VMware vRealize Operations Manager
And in case you’re wondering: Yes, that are 2x Philips Hue Light Strips. Because of reasons.
Also, here are some pictures: