Designing a vSAN cluster from scratch poses a lot of challenges. One of the challenges I encountered is to ensure that resources (memory, CPU and storage) in the cluster gets utilized evenly, and that you for instance don't run out of memory while having plenty of remaining storage capacity.
I find the following a good approach to achieve this.
1. VM Resource Utilization
Get the combined VM storage, memory and CPU utilization of the VMs that will run on the vSAN cluster.
For instance, if you had 100 VMs, each with an average utilization of 150GB disk, 12GB RAM and 2 GHz CPU, the total resource utilization would be:
2. Resource Utilization Ratios
The next step will be to calculate the resource utilization ratio between memory and CPU, and between memory and storage.
Mem-CPU Ratio = Total Mem / Total CPU
Stor-Mem Ratio = Total Storage / Total Mem
In this example the Mem-CPU ratio would be 6, and the Stor-Mem ratio would be 13.5:
1200/200 = 6
16,200/1200 = 13.5
3. Host CPU and Memory
To figure out how much CPU and memory each host should have divide the assumed memory of the host with the Mem-CPU ratio.
Host CPU = Host memory / Mem-CPU ratio
In this example the ratio is 6, i.e. 6 times as much RAM in GB as CPU in GHz. If we aim to have 256GB of RAM in the host, this would equate to 42.7 GHz of CPU.
256GB/6 = 42.7 GHz
42.7 GHz per host is quite high, and it is also generally easier to upgrade memory than CPU. Therefore to allow for increased CPU resources we might want to overspec the CPU slightly and for example use a server with dual Intel E5-2680 v3 (12 core 2.5GHz), with a total of 60 GHz.
To ensure we easily can upgrade memory if needed, we might decide to utilize 8 * 32GB DIMMs. If we use a standard 2U server such as an HP Proliant DL380 G9, means we can double the amount of RAM without losing memory speed.
4. Usable Host Disk Capacity
The amount of usable disk capacity in each host also depends on the amount of memory in the host and is calculated by multiplying the total host memory with the Stor-Mem Ratio.
Usable Host disk space = Host Memory * Stor-Mem Ratio
If each host was configured with 256GB of RAM and we accepted that we wouldn't use more than 80% of the total capacity, the usable amount of memory in a host would therefore be 205GB. If using the example of a Stor-Mem ratio of 13.5 the total usable disk space of the host would be 2768GB:
205GB * 13.5 = 2768GB
5. Host Raw Disk Capacity
The raw disk capacity of each host would depend on both the max percentage of disk utilization and the expected failure to tolerate (FTT) storage policy of the cluster (encryption & deduplication might have to be considered in an all flash configuration).
Raw disk capacity = usable disk capacity * (FTT + 1) / max utilization
In the example I want to limit the disk utilization to 70% and and expect a failure to tolerate policy of 1. The raw storage capacity for each host would then be 7909GB:
2768GB * (1 + 1) / 0.7 = 7909GB
6. Capacity Disk Layout
Knowing the required raw storage capacity per host, makes it much easier to find a suitable capacity disk configuration.
In the example of having 7909GB raw storage capacity per host. A solution for a hybrid vSAN this could be 2 disk groups with 4 * 1.2TB SAS disk in each.
7. Host Configuration
We now have the basic layout of the host (I'm ignoring the cache which is covered in other blogs). Which in our example would be:
CPU: Dual E5-2680 v3
RAM: 256GB (8 * 32GB)
Capacity Disks: 8 * 1.2 TB SAS disks.
8. Hosts in the Cluster
Now when we know how each host will look like we figure out how many hosts are required in the cluster.
Total Hosts = Total Memory/Host Memory
In this example it would be:
1200/205 = 5.85 (rounded up to 6)
Adjusted for high availability, this would become 7 hosts for N+1, and we can be confident that all the resources will be evenly utilized in this cluster.