{"id":28508,"date":"2023-05-26T10:11:20","date_gmt":"2023-05-26T08:11:20","guid":{"rendered":"https:\/\/blog.miniserver.it\/?p=28508"},"modified":"2023-05-30T16:12:09","modified_gmt":"2023-05-30T14:12:09","slug":"size-3-node-hyper-converged-proxmox-cluster","status":"publish","type":"post","link":"https:\/\/blog.miniserver.it\/en\/proxmox-ve\/size-3-node-hyper-converged-proxmox-cluster\/","title":{"rendered":"How to Size a 3 Node Hyper-Converged Proxmox VE Cluster"},"content":{"rendered":"<div class=\"fusion-fullwidth fullwidth-box fusion-builder-row-1 fusion-flex-container has-pattern-background has-mask-background nonhundred-percent-fullwidth non-hundred-percent-height-scrolling\" style=\"--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;\" ><div class=\"fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap\" style=\"max-width:1123.2px;margin-left: calc(-4% \/ 2 );margin-right: calc(-4% \/ 2 );\"><div class=\"fusion-layout-column fusion_builder_column fusion-builder-column-0 fusion_builder_column_1_1 1_1 fusion-flex-column\" style=\"--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;\"><div class=\"fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column\"><div class=\"fusion-text fusion-text-1\"><p>In this article we will discuss how to correctly size the hardware resources of a 3 node <a href=\"https:\/\/blog.miniserver.it\/en\/proxmox\/\" target=\"_blank\" rel=\"noopener noreferrer\"><b>Proxmox VE<\/b><\/a> hyper-converged cluster and how to get an estimate for the best possible solution.<\/p>\n<p>The cluster we&#8217;ll be sizing below consists of 3 nodes and 2 switches.<\/p>\n<p>It&#8217;s very important to understandi how the hyperconverged cluster works.<\/p>\n<\/div><\/div><\/div><div class=\"fusion-layout-column fusion_builder_column fusion-builder-column-1 fusion_builder_column_1_1 1_1 fusion-flex-column\" style=\"--awb-padding-right:20px;--awb-padding-left:20px;--awb-bg-color:#f1f4f6;--awb-bg-color-hover:#f1f4f6;--awb-bg-blend:overlay;--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:0px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;\"><div class=\"fusion-column-wrapper fusion-flex-justify-content-flex-start fusion-content-layout-column\"><div class=\"fusion-title title fusion-title-1 fusion-sep-none fusion-title-text fusion-title-size-two\" style=\"--awb-margin-top:20px;--awb-margin-bottom:20px;--awb-margin-top-small:10px;--awb-margin-right-small:0px;--awb-margin-bottom-small:10px;--awb-margin-left-small:0px;\"><h2 class=\"fusion-title-heading title-heading-left\" style=\"margin:0;\">Sizing of the 3 Node Proxmox VE Cluster<\/h2><\/div><div class=\"awb-toc-el awb-toc-el--1\" data-awb-toc-id=\"1\" data-awb-toc-options=\"{&quot;allowed_heading_tags&quot;:{&quot;h3&quot;:0},&quot;ignore_headings&quot;:&quot;&quot;,&quot;ignore_headings_words&quot;:&quot;Il dimensionamento del cluster Prxmox VE a 3 nodi&quot;,&quot;enable_cache&quot;:&quot;yes&quot;,&quot;highlight_current_heading&quot;:&quot;no&quot;,&quot;hide_hidden_titles&quot;:&quot;yes&quot;,&quot;limit_container&quot;:&quot;post_content&quot;,&quot;select_custom_headings&quot;:&quot;&quot;,&quot;icon&quot;:&quot;fa-flag fas&quot;,&quot;counter_type&quot;:&quot;none&quot;}\" style=\"--awb-margin-bottom:20px;--awb-margin-left:30px;\"><div class=\"awb-toc-el__content\"><ul class=\"awb-toc-el__list awb-toc-el__list--0\"><li class=\"awb-toc-el__list-item\"><a class=\"awb-toc-el__item-anchor\" href=\"#toc_1_Il_ragionamento_alla_base_del_progetto\">1. The logic behind the project<\/a><\/li><li class=\"awb-toc-el__list-item\"><a class=\"awb-toc-el__item-anchor\" href=\"#toc_2_I_parametri_indispensabili_per_dimensionate_il_cluster\">2. Essential parameters for cluster sizing<\/a><\/li><li class=\"awb-toc-el__list-item\"><a class=\"awb-toc-el__item-anchor\" href=\"#toc_2_I_parametri_indispensabili_per_dimensionate_il_cluster\">3. Cluster hardware upgrade<\/a><\/li><li class=\"awb-toc-el__list-item\"><a class=\"awb-toc-el__item-anchor\" href=\"#toc_4_La_scelta_dello_switch\">4. Choosing the switch<\/a><\/li><li class=\"awb-toc-el__list-item\"><a class=\"awb-toc-el__item-anchor\" href=\"#toc_5_Scelta_della_tecnologia_disco\">5. Choosing the disks<\/a><\/li><li class=\"awb-toc-el__list-item\"><a class=\"awb-toc-el__item-anchor\" href=\"#toc_6_Come_ottenere_un_preventivo\">6. How to request a quote<\/a><\/li><li class=\"awb-toc-el__list-item\"><a class=\"awb-toc-el__item-anchor\" href=\"#toc_7_Cluster_a_2_nodi_iperconvergenti\">7. 2 Node Hyper-Converged Cluster<\/a><\/li><\/ul><\/div><\/div><\/div><\/div><div class=\"fusion-layout-column fusion_builder_column fusion-builder-column-2 fusion_builder_column_1_1 1_1 fusion-flex-column\" style=\"--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;\"><div class=\"fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column\"><div class=\"fusion-text fusion-text-2\"><h3 id=\"toc_1_Il_ragionamento_alla_base_del_progetto\">1. The logic behind the project<\/h3>\n<p><span style=\"background-color: var(--awb-bg-color-hover); color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform);\">A few introductory remarks before starting:<\/span><\/p>\n<ul>\n<li>The cluster consists of three servers (also called nodes or hypervisors).<\/li>\n<li>The hyper-converged storage space, at the core of the 3 node cluster, is calculated differently than CPU and RAM resources.<\/li>\n<li>Cluster load is represented by a Virtual Machine (VM) and a Container (CT).<\/li>\n<li>The cluster must continue to operate without service interruption even if a node and\/or switch goes down.<\/li>\n<\/ul>\n<p>Sizing will be done considering a fundamental characteristic:<\/p>\n<p>A cluster makes sense because it has Fault Tolerance characteristics.<\/p>\n<p>Meaning a cluster consisting of 3 nodes and 2 switches <b>will work without interruption, even in the event of a node or switch failure.<\/b><\/p>\n<p>If you are unable to run VMs and CTs on two of the three nodes due to lack of hardware resources, the the cluser concept is lost.<\/p>\n<p>Such scenario can and should be avoided with correct hardware sizing.<\/p>\n<p>These Fault Tolerance characteristics indicate that all hardware requirements of the virtual machines and containers will have to run on 2 nodes and 1 switch.<\/p>\n<p>This means that, in our example, you can afford losing one node and one switch without affecting how the resources work.<\/p>\n<\/div><div class=\"fusion-text fusion-text-3\"><h3 id=\"toc_2_I_parametri_indispensabili_per_dimensionate_il_cluster\">2. Essential parameters for cluster sizing<\/h3>\n<p>First, we need to retrieve the information necessary to carry out the sizing.<\/p>\n<p>The essential parameters to size the cluster are as follows:<\/p>\n<ul>\n<li>A description of the resources (VM and\/or CT). We give the resource a name as in the table below.<\/li>\n<li>Type of operating system: Windows \/ Linux, etc\u2026<\/li>\n<li>Disk space in GB.<\/li>\n<li>Number of physical cores.<\/li>\n<li>RAM in GB<\/li>\n<\/ul>\n<p>How is hyper-converged space calculated?<\/p>\n<p><span style=\"background-color: var(--awb-bg-color-hover); color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform);\">For enstanace, if we have 3 nodes and each node has 4TB of total storage space, the total hyper-converged space of the cluster will be 4TB.<\/span><\/p>\n<p>This is because in a 3 node cluster, the replication factor is 3. Therefore data, by definition of hyper-convergence, is replicated three times within the cluster.<\/p>\n<p>Core and RAM calculation is different, as the resources of each node add up.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-28575\" src=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example.png\" alt=\"How to Size a 3 Node Hyper-Converged Proxmox VE Cluster - Example\" width=\"852\" height=\"464\" srcset=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example-200x109.png 200w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example-400x218.png 400w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example-600x327.png 600w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example-621x338.png 621w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example-768x418.png 768w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example-800x436.png 800w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/How-to-Size-a-3-Node-Hyper-Converged-Proxmox-VE-Cluster-Example.png 852w\" sizes=\"(max-width: 852px) 100vw, 852px\" \/><\/p>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">Let&#8217;s consider the cost in terms of RAM and CPU resources of the hypervisor which we have estimated at more or less 10%.<\/span><\/p>\n<p>We then estimate a 50% load growth. We calculate a growth in demand of more or less 50% from the date of installation until the end of life of the solution.<span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\"><br \/><\/span><\/p>\n<p>You can make your own estimations by entering data that best satisfies your case.<\/p>\n<p>We increase Cores and RAM enough to run the VMs and CTs on 2 of the 3 nodes of the cluster.<\/p>\n<p>In our example, we take Core and RAM, respectively 41 and 254 and divide by a factor of two.<\/p>\n<p>After rounding up, we get the data present on the &#8220;Single Node Resources&#8221; row. We thus obtain the resources of the &#8220;Single Node Resources&#8221;.<\/p>\n<p>Note that the same does not apply to storage, as the total space is already replicated 3 times.<\/p>\n<p>The size we see on the &#8220;Single Node Resources&#8221; line regarding disk space will already be that of the single node.<\/p>\n<p>To obtain optimal disk performance, note that Ceph works very well with at least 4 disks per node.<\/p>\n<p>However, operation is guaranteed even with 2 discs.<\/p>\n<\/div><div class=\"fusion-text fusion-text-4\"><h3 id=\"toc_2_I_parametri_indispensabili_per_dimensionate_il_cluster\">3. Cluster hardware upgrade<\/h3>\n<p>Now let&#8217;s look at hardware upgrades and how they impact sizing.<\/p>\n<p>In our previous example, we estimated a 50% increase in hardware resources for the same system load request.<\/p>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">What could happen if this 50% estiamte is not done from the start? <\/span><\/p>\n<p>Se non prevediamo questo 50% fin da subito, cosa potrebbe succedere in futuro?<\/p>\n<p>We have two scenarios:<\/p>\n<p><b>Scenario 1:<\/b><\/p>\n<p>You can increase hardware resources by increasing either disk space and\/or RAM by simply adding parts.<\/p>\n<p>As for the disks and RAM, the process is trivial.<\/p>\n<p>When it comes to CPU cores, it is not that simple. You need to unmount the CPU and mount compatible one.<\/p>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">It is feasible but technically more complex.<\/span><\/p>\n<p><b>Scenario 2:<\/b><\/p>\n<p>Increase the number of nodes.<\/p>\n<p>This is a powerful cluster feature since all you need to do is to add a node to the cluster to expand storage space, RAM and CPU.<\/p>\n<p>Therefore, correctly providing networking (switch) is crucial, since it is an integral part of the system&#8217;s performance.<\/p>\n<\/div><div class=\"fusion-text fusion-text-5\"><h3 id=\"toc_4_La_scelta_dello_switch\">4. Choosing the switch<\/h3>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">In addition to connecting virtual machines or containers to the world, <\/span><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">switches <\/span><span style=\"background-color: var(--awb-bg-color-hover); color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform);\">also manage the cluster&#8217;s hyper-convergence.<\/span><\/p>\n<p>Keep in mind that every time you write data on a VM or a CT, it passes through the networking (network cards and switches), which cannot and must never be a bottleneck.<\/p>\n<p>For this purpose, right from the beginning, you need to chose a switch that will be suitable for both current and future needs.<\/p>\n<\/div><div class=\"fusion-text fusion-text-6\"><h3 id=\"toc_5_Scelta_della_tecnologia_disco\" style=\"color: #747474;\">5. Choosing the disks<\/h3>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">The choice of storage tends to be that of choosing between SSD and NVMe.<\/span><\/p>\n<p>We receive many requests for clusters with rotative SATA or SAS type hyper-converged storage, due to low cost of them.<\/p>\n<p>Unfortunately, due to the poor performances, a hyper-converged solution with these technologies is not feasible.<\/p>\n<p>One of the main reasons is that during operation, OSD realignments can occur which cause performance peaks.<\/p>\n<p>During such peaks the spinning disk is not fast enough to provide enough I\/O to the VM and CT to perform required hyper-convergence tasks.<\/p>\n<p>All of this causes Ceph errors, generating cluster stability and performance issues.<\/p>\n<p>Spinning disk technology has problems. For this reason its use is strongly discouraged.<\/p>\n<\/div><div class=\"fusion-text fusion-text-7\"><h3 id=\"toc_6_Come_ottenere_un_preventivo\">6. How to request a quote<\/h3>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">Once the hardware characteristics of the single node have been defined, we can proceed to request an estimate to have an fiancial quantification of the investment.<\/span><\/p>\n<p>If you are not certain about the choices, we suggest leaving the default selection.<\/p>\n<p>&nbsp;<\/p>\n<p>In any case, each quote is verified before being sent out.<\/p>\n<p><span lang=\"it-IT\">Based on our example&#8217;s speficications we&#8217;re working with the <a href=\"https:\/\/www.miniserver.store\/proxmox-cluster-s1b\" target=\"_blank\" rel=\"noopener noreferrer\"><b>3 Node Hyper-Converged Proxmox VE Cluster &#8211; S1B<\/b><\/a><b><\/b><b><\/b><b><\/b><\/span><\/p>\n<\/div><div class=\"fusion-text fusion-text-8\"><p>We said that about 21 Cores are needed. In this case we round them down.<\/p>\n<\/div><div class=\"fusion-image-element \" style=\"--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);\"><span class=\" fusion-imageframe imageframe-none imageframe-1 hover-type-none\"><img decoding=\"async\" width=\"743\" height=\"169\" title=\"CPU\" src=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/CPU-.png\" alt class=\"img-responsive wp-image-28578\" srcset=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/CPU--200x45.png 200w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/CPU--400x91.png 400w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/CPU--600x136.png 600w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/CPU-.png 743w\" sizes=\"(max-width: 640px) 100vw, 743px\" \/><\/span><\/div><div class=\"fusion-text fusion-text-9\" style=\"--awb-margin-top:40px;\"><p>Then we move on to RAM memory. The number that comes closest is 128GB<\/p>\n<\/div><div class=\"fusion-image-element \" style=\"--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);\"><span class=\" fusion-imageframe imageframe-none imageframe-2 hover-type-none\"><img decoding=\"async\" width=\"742\" height=\"147\" title=\"RAM\" src=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/RAM-.png\" alt class=\"img-responsive wp-image-28581\" srcset=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/RAM--200x40.png 200w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/RAM--400x79.png 400w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/RAM--600x119.png 600w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/RAM-.png 742w\" sizes=\"(max-width: 640px) 100vw, 742px\" \/><\/span><\/div><div class=\"fusion-text fusion-text-10\" style=\"--awb-margin-top:40px;\"><p>Since this is a production system we choose the annual subscription &#8211; <a href=\"https:\/\/www.miniserver.store\/licensing\/proxmox-ve-community-subscription-1-cpu-year\" target=\"_blank\" rel=\"noopener noreferrer\"><b>Proxmox VE Community,<\/b><\/a><\/p>\n<\/div><div class=\"fusion-image-element \" style=\"--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);\"><span class=\" fusion-imageframe imageframe-none imageframe-3 hover-type-none\"><img decoding=\"async\" width=\"736\" height=\"146\" title=\"Pre-installed operating system\" src=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Pre-installed-operating-system.png\" alt class=\"img-responsive wp-image-28584\" srcset=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Pre-installed-operating-system-200x40.png 200w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Pre-installed-operating-system-400x79.png 400w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Pre-installed-operating-system-600x119.png 600w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Pre-installed-operating-system.png 736w\" sizes=\"(max-width: 640px) 100vw, 736px\" \/><\/span><\/div><div class=\"fusion-text fusion-text-11\" style=\"--awb-margin-top:40px;\"><p>The OS will be installed on two Raid 1 128GB drives. This is already the default.<\/p>\n<\/div><div class=\"fusion-image-element \" style=\"--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);\"><span class=\" fusion-imageframe imageframe-none imageframe-4 hover-type-none\"><img decoding=\"async\" width=\"741\" height=\"149\" alt=\"Operating System Storage\" title=\"Operating System Storage\" src=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Operating-System-Storage.png\" class=\"img-responsive wp-image-28587\" srcset=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Operating-System-Storage-200x40.png 200w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Operating-System-Storage-400x80.png 400w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Operating-System-Storage-600x121.png 600w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Operating-System-Storage.png 741w\" sizes=\"(max-width: 640px) 100vw, 741px\" \/><\/span><\/div><div class=\"fusion-text fusion-text-12\" style=\"--awb-margin-top:40px;\"><p>Storage is around 6TB. We choose the closest denomination, in this case 8 TB.<\/p>\n<\/div><div class=\"fusion-image-element \" style=\"--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);\"><span class=\" fusion-imageframe imageframe-none imageframe-5 hover-type-none\"><img decoding=\"async\" width=\"736\" height=\"146\" alt=\"Storage Ceph hyperconverged space\" title=\"Storage Ceph hyperconverged space\" src=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Storage-Ceph-hyperconverged-space-.png\" class=\"img-responsive wp-image-28590\" srcset=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Storage-Ceph-hyperconverged-space--200x40.png 200w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Storage-Ceph-hyperconverged-space--400x79.png 400w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Storage-Ceph-hyperconverged-space--600x119.png 600w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Storage-Ceph-hyperconverged-space-.png 736w\" sizes=\"(max-width: 640px) 100vw, 736px\" \/><\/span><\/div><div class=\"fusion-text fusion-text-13\" style=\"--awb-margin-top:40px;\"><p>As mentioned above, networking is an integral part of the cluster, therefore data written on a virtual machine, wherever it is located, is written in real time on all three. D<span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">ata will then pass from the network.<\/span><\/p>\n<p>From testing, and from data easily available online, we can easily see that SSD or NVMe disks can easily saturate a 10 Gbit connection.<\/p>\n<p>For this reason the cluster must be equiped with at least 25 Gbit cards and switches to avoid bottlenecks.<\/p>\n<p>The S1B cluster is equipped with SSD disks and often 25 GB network cards.<\/p>\n<p>We&#8217;ll use two redundant switches. Each node will be connected to switch 1 and switch 2 to ensure redundancy.<\/p>\n<p>Switches will be stacked together. Considering 25GB hyper-converged networking between nodes, we&#8217;lll use a 40Gbit stack to avoid bottlenecks.<\/p>\n<p>Switches will need to support the Stack protocol as well as node NIC aggregation protocols.<\/p>\n<p>In this will enable you to take advantage of all the speed of the installed hardware and total redundancy, both on the switches and on the NICs.<\/p>\n<\/div><div class=\"fusion-image-element \" style=\"--awb-caption-title-font-family:var(--h2_typography-font-family);--awb-caption-title-font-weight:var(--h2_typography-font-weight);--awb-caption-title-font-style:var(--h2_typography-font-style);--awb-caption-title-size:var(--h2_typography-font-size);--awb-caption-title-transform:var(--h2_typography-text-transform);--awb-caption-title-line-height:var(--h2_typography-line-height);--awb-caption-title-letter-spacing:var(--h2_typography-letter-spacing);\"><span class=\" fusion-imageframe imageframe-none imageframe-6 hover-type-none\"><img decoding=\"async\" width=\"740\" height=\"313\" alt=\"Networking dedicated to Ceph\" title=\"Networking dedicated to Ceph\" src=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Networking-dedicated-to-Ceph.png\" class=\"img-responsive wp-image-28592\" srcset=\"https:\/\/blog.miniserver.it\/wp-content\/uploads\/Networking-dedicated-to-Ceph-200x85.png 200w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Networking-dedicated-to-Ceph-400x169.png 400w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Networking-dedicated-to-Ceph-600x254.png 600w, https:\/\/blog.miniserver.it\/wp-content\/uploads\/Networking-dedicated-to-Ceph.png 740w\" sizes=\"(max-width: 640px) 100vw, 740px\" \/><\/span><\/div><div class=\"fusion-text fusion-text-14\" style=\"--awb-margin-top:40px;\"><p>At the end of the configuration, click on &#8220;Add to Quote&#8221;.<\/p>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">We will be happy to help you with the configuration and in choosing the model that best suits your needs.<\/span><\/p>\n<\/div><div class=\"fusion-text fusion-text-15\"><h3 id=\"toc_7_Cluster_a_2_nodi_iperconvergenti\">7. 2 Node Hyper-Converged Cluster<\/h3>\n<p>There are articles and videos online discussing hyper- convergence with 2 node clusters.<\/p>\n<p>Currently, hyper-convergence with Proxmox VE is not feasible with 2nodes.<\/p>\n<p>We strongly doubt the efficiency and resilience of solutions from other brands that market 2 node solutions as technically they do not seem feasible solutions.<\/p>\n<p>The solutions we tested turned out to be unsuitable because, while sometimess providing the promised redundancy, in some failure scenarios they were unable to &#8220;understand&#8221; the state of the nodes, thus causing serious system malfunctions.<\/p>\n<p><span style=\"color: var(--awb-text-color); font-family: var(--awb-text-font-family); font-size: var(--awb-font-size); font-style: var(--awb-text-font-style); font-weight: var(--awb-text-font-weight); letter-spacing: var(--awb-letter-spacing); text-align: var(--awb-content-alignment); text-transform: var(--awb-text-transform); background-color: var(--awb-bg-color-hover);\">There are also technical limitations therefore there is no reliable solution that can be considered suitable for a 2 node production environment.<\/span><\/p>\n<\/div><\/div><\/div><\/div><\/div>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":11,"featured_media":28573,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[129],"tags":[],"class_list":["post-28508","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-proxmox-ve"],"_links":{"self":[{"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/posts\/28508","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/comments?post=28508"}],"version-history":[{"count":57,"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/posts\/28508\/revisions"}],"predecessor-version":[{"id":28594,"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/posts\/28508\/revisions\/28594"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/media\/28573"}],"wp:attachment":[{"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/media?parent=28508"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/categories?post=28508"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.miniserver.it\/en\/wp-json\/wp\/v2\/tags?post=28508"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}