Originally posted on the VMware vSphere Blog.
I have a confession.
My data center kit has been using too much energy.
Having kit available at my disposable is great, but I have been wasting this resource when it’s not required by my workloads. And if there’s one thing I try to be conscious of, it’s energy consumption. Just ask my kids who I chase from room to room turning off lights, screens, and the lot when they aren’t using them.
But why not in the data center? Did you know that hosts typically use 60%+ of their peak power when idle?
Until recently, I had overlooked configuring my kit to use the vSphere Distributed Power Management (“DPM”) feature to manage power consumption and save energy.
With the release of vSphere 6.0 it’s a good time to review and take deeper look into the capabilities and benefits of this feature.
VMware vSphere Distributed Power Management is a feature included with vSphere Enterprise and Enterprise Plus editions that dynamically optimizes cluster power consumption based on workload demands. When host CPU and memory resources are lightly used, DPM recommends the evacuation of workloads and powers-off of ESXi hosts. When CPU or memory resource utilization increases for workloads or additional host resources are required, DPM powers on a required set of hosts back online to meet the demand of HA or other workload-specific contraints by executing vSphere Distributed Resource Scheduler (“DRS”) in a “what-if” mode. DRS will ensure host power recommendations are consistent with the cluster constraints and resources being managed by the cluster.
Beneath the covers there are key challenges that DPM addresses to enable effective power-savings capabilities:
Once DPM determines the number of hosts needed to satisfy all workloads and relevant constraints, and DRS has distributed virtual machines across hosts to maintain resource allocation constraints and objectives, each powered-on host is free to handle its power management
When a host is powered-off by DPM, they are marked in vCenter Server in “standby” mode indicating that they are powered-off but available to be powered-on when required. The host icon is updated with a crescent moon overlay symbolizing a “sleeping” state for the host.
DPM can awaken hosts from the standby mode using one of three power management options:
Each protocol requires its own hardware support and configuration. If a host does not support any of these protocols it cannot be put into standby by DPM. If a host supports multiple protocols, they are used in the following order: IPMI, iLO, WOL. This article is focused on the use of the first two.
IPMI is a hardware-level specification and Hewlett-Packard iLO is an embedded server management technology. Each of them describes and provides an interface for remotely monitoring and controlling computers. Both require a hardware Baseboard Management Controller (“BMC”) to provide a gateway for accessing hardware control functions, and allow the interface to be accessed from a remote system using serial or LAN connections. The BMC is powered-on even when the host itself is powered-off. And, if properly enabled, the BMC can respond to remote power-on commands.
To use IPMI or iLO and with vSphere Distributed Power Management, you must properly configure the host BMC. These steps may vary according to vendor and model. Now, with IPMI, you must also ensure that the BMC LAN channel is configured to be always available and to allow operator-privileged commands. Keep in mind that some system IPMI require that enabling “IPMI over LAN” in the BIOS and the specification an IPMI account.
DPM using only IPMI supports MD5- and plaintext-based authentication. MD2-based authentication is not supported. vCenter Server uses MD5 if a host’s BMC reports that it is supported and enabled for the operator role. Otherwise, plaintext-based authentication is used if the BMC reports it is supported and enabled. If neither MD5 nor plaintext-based authentication is enabled, IPMI cannot be used with the host and vCenter Server attempts to use the WoL protocol.
Before enabling Power Management for hosts in a vSphere cluster, it is necessary to gather the username and password for IP and MAC Address of the BMC.
In this post, the hosts in my Management Cluster use HP iLO 4 and I have captured both the IP and MAC Address used for their interfaces. I have also designated a user that is allowed to perform only “Virtual Power and Reset” operations. The user has been restricted from all other roles and operations.
Now, let’s configure all the hosts in the Management Cluster for Power Management.
Before enabling DPM on the cluster, it’s a good idea to validate that vCenter Server can communicate with each host’s BMC by individually testing placing it in standby mode and then powering-on the host through the vSphere Web Client.
Get-VMHost -Name <Hostname> | Suspend-VMHost -Confirm:$false
Get-VMHost -Name <Hostname> | Start-VMHost -Confirm:$false
As mentioned previously, DPM leverages DRS when to migrates virtual machines away from hosts that are ready to be powered-off and marked as stand-by. Therefore, DPM requires that DRS to be enabled on all clusters in which it is enabled. Let’s look at how to enable DPM in this Management Cluster.
DemandCapacityRatioTarget = utilization target of the host - by default it is 63%
DemandCapacityRatioToleranceHost = variation around utilization target - by default it is 18%
Default DPM settings can be changed in the DRS Advanced Settings. and can be set between 40% and 90% for DemandCapacityRatioTarget and between 10% and 40% for DemandCapacityRatioToleranceHost. Two time intervals are used by DPM when evaluating power-on and power-off recommendations. For power-on the period is 300 seconds (5 Minutes) and power off is 2400 seconds (40 Minutes.) Therefore, DPM considers more important responses to increased load. Note however that a sudden increase in the load will be be considered by DPM only after 5 minutes and will initiate a resolution after the host complete its power-on operation in the cluster. The default values can be changed by setting parameters VmDemandHistorySecsHostOn (default 300 seconds) and VmDemandHistorySecsHostOff (default 2400 seconds) to a value between 0 and 3600 seconds. DPM always ensure that at least one host will be running within the DPM-enabled cluster. The settings MinPoweredOnCpuCapacity (default 1 MHz) and MinPoweredOnMemCapacity (default 1 MB) are used to control how many hosts are remain powered-on. There default values ensure minimum one host is up and can be changed. For example if the cluster has hosts configured with 24 GHz and 128 GB setting the parameters 24001 MHz and 131073 MB will reserve 2 hosts running at all time. Even with default values, when HA cluster is enabled, DPM will leave 2 hosts powered on to provide fail over resources in case of one host failing.
Utilization range = DemandCapacityRatioTarget +/- DemandCapacityRatioToleranceHost
DrsEnteringStandbyModeEvent = Entering Standby mode (about to power off host)
DrsEnteredStandbyModeEvent = Successfully entered Standby mode (host power off succeeded)
DrsExitingStandbyModeEvent = Exiting Standby mode (about to power on the host)
DrsExitedStandbyModeEvent = Successfully exited Standby mode (power on succeeded)Modern data centers use tools to monitor the state and health of data center objects. If you use monitoring software, other than vCenter Server, that triggers alarms when ESXi hosts are powered-off unexpectedly, you might have a situation where false alarms are generated when DPM places a host into standby mode. To alleviate false positives, work with your vendor to deploy a version of the monitoring software that is integrated with vCenter Server and triggers events indicating that a host has entered DPM-initiated Standby mode. Alternatively you may use vCenter Server as your host monitoring solution as it is inherently aware of DPM and does not trigger these false positive alarms. Do you have an opportunity to reduce power consumption in your data center? Examine the ability to streamline your data center operations and save energy with vSphere DPM. - - - -