Data Growth In Always On World Driving Data Center Demand

At the 2010 Techonomy Conference, Google’s then-CEO Eric Schmidt noted that every two days, people create as much information as civilization did from the dawn of time until 2003 — an estimated five exabytes every two day. The explosion of corporate, academic, and governmental data, combined with explosive growth in user-generated content (Twitter, Facebook, YouTube, etc.), is changing the applications landscape, as companies and customers begin to understand the potent mix that online data, high-performance computing, and high-bandwidth networking create. We are already an Always On society, and the future only becomes even more On.

As semiconductor miniaturization enters the deep submicron era, the density of electrical/mechanical loads continues to increase in server farms. Virtualization has helped to increase processor utilization leading to even greater demands on cooling services. And the unique demands of always on computing create new challenges in mirroring, distributed content distribution networks, and cost-efficient peak load servicing.

At BRUNS-PAK, we recognize that the evolution of the data center, and with it our design-build methodologies, is a never-ending process. As the information technology industry continues to evolve and strive ultrareliable performance complemented by increasingly efficient operation and scalable deployment, only BRUNS-PAK has committed the resources to ensure that our technical team will remain ready to design and deliver “state-of-the- art” facilities even as the “art” changes.

High Impact Measures to Boost Data Center Efficiency (Part 1)

With Data Center energy consumption at an all time high, maintaining the lowest possible total cost of ownership has become increasingly difficult.  We’ve detailed some high impact measures to help improve efficiency, and reduce power and cooling requirements to create a greener, more cost effective Data Center.

The first step in energy-efficiency planning is measuring current energy usage.  The power system is a critical element in the facilities infrastructure, and knowing where that energy is used and by which specific equipment is essential when creating, expanding, or optimizing a Data Center.

In order to understand how energy efficiency measures affect energy consumption in the Data Center, a baseline needs to be established for the current energy used. There are currently two primary metrics being used by a number of organizations such as the Green Grid to promote the notion of measuring Data Center energy efficiency. The first is Power Usage Effectiveness (PUE) which is defined as the total facility power consumed divided by the IT equipment power consumption. The second metric is PUE’s reciprocal known as Data Center Infrastructure Efficiency (DCiE) which is defined as the IT equipment power consumed divided by the total facility power consumption.

Total facility power is defined as power measured from the utility meter or switch gear solely dedicated to the operation of the Data Center infrastructure in the building if the building is a shared facility with other functions. This includes power consumed by electrical equipment such as switchgear, UPSs (uninterruptible power system) and batteries, PDUs (power distribution units), and stand-by generators. Mechanical equipment dedicated to the HVAC needs of the Data Center such as CRACs (computer room air conditioning units), chillers, DX (direct expansion) air handler units, drycoolers, pumps, and cooling towers. IT equipment power includes the loads associated with IT processes including server, storage, network, tape and other processing equipment fed through Data Center infrastructure support equipment such as PDUs, RPPs (remote power panels), or other distribution means fed from a UPS.

To collect the information noted above, an effective building management system (BMS) should be employed to help collect, categorize, and trend the data gathered. Most systems offered by BMS providers such as Johnson Controls, Andover, Automated Logic, Honeywell, Siemens, and others can allow monitoring of energy consumption for both the IT equipment and the associated infrastructure equipment serving the Data Center. Metering and other DCPs (data collection points) should be provided at all switchgear relating to power and mechanical needs of the Data Center. Also metering should be provided at the output side of the UPS modules or better yet the PDUs. This will provide the energy consumption rates of both the facility power and IT equipment power.

The types of electrical monitoring which can be employed to measure this type of information can be broken down into three basic forms:

  • Amperage-only monitoring
  • Estimated Wattage monitoring
  • True RMS Wattage monitoring

Amperage-only and Estimated Wattage monitoring means can be flawed in the information they provide due to the inaccuracies of measuring the sine wave and its form. Should a sine wave be produced inaccurately, as many double conversion UPS systems do, averaging means of formulating power consumption can prove to be flawed. True RMS Wattage monitoring provides a much more accurate means of understanding the idiosyncrasies of power consumption relating to data processing power sources. BMS systems which employ measures such as wave form capture sampling using real time updating provide a very high degree of accuracy. It should be pointed out that this type of monitoring can be expensive at implementation based on the number of locations it is determined to be used. Should the decision be made to measure power at the distribution level of PDUs and CRAC units to determine power consumption, the cost at this level can be higher than if monitoring was to be placed at the distribution panel boards feeding these types of devices. As long as all the IT equipment and associated infrastructure equipment is being fed from a singular (or dual) location, this monitoring may be far less expensive while still providing nearly the same information for the distributed systems required out in the Data Center.

Traditional Data Centers which are not currently enacting any type of energy efficiency measures are operating with an average PUE of over 3.  A Data Center which is actively pursuing energy efficient measures can achieve much lower ratings, and in return realize substantial energy savings.

High Impact Measures to Boost Data Center Efficiency (Part 2)

While typical energy audits focus on the mechanical and electrical infrastructure, in data
centers the facility framework is only one factor in the cost equation. Often times improving in other areas can be even more rewarding. For example, consideration of the actual kilowatts consumed by servers and other IT hardware is crucial when examining energy efficiency in a data center.

Data processing equipment accounts for most of the energy consumption in a data center, and because of this, facility executives really need to start by thinking ‘inside the box’.  Best practices in the type, usage, and configuration of deployment all can significantly reduce the overall energy needs for this equipment.

Pull the Plug on Idle Servers

It’s a simple concept really, if it’s not doing anything, unplug it.  However, in many data centers, up to 15% of the servers should be decommissioned and yet are left running for no other reason than lack of drive to clean up outdated equipment.  Some estimates indicate that the cost of each idle server can exceed $1,000 annually when considering total data center energy usage.  That’s a lot of wasted capital!  Addressing the issue can have an immediate impact on the bottom line.

The solution is to establish a rigorous program to decommission obsolete hardware.

Maintaining an asset management database is a necessity to help enterprises ensure that they are consuming resources efficiently. This database should contain accurate, up-to-date information on server location and configuration, enabling IT staff to easily identify variables of power, cooling, and available rack space when planning future server and storage deployments and identifying potential systems to retire.

Upgrade to Energy Efficient Servers

Another simple measure to reducing energy consumption is to buy more energy-efficient
servers.  The bulk of IT departments ignore energy efficiency ratings when selecting new hardware, focusing on performance and up-front costs rather than total cost of ownership.  However, if just one server uses 50 watts less than another it will equate to a savings of more than $250 over a three year period, and an even more profound savings of $1,500+ on facility infrastructure expenditures can be realized.

Data processing equipment all rely on power supplies to take incoming power to the
device and distribute it accordingly throughout its internal components as
required. These power supplies are typically specified by the manufacturer to
provide for the worse case conditions of the device under a maximized
configuration. In the past, these power supplies typically were rated far
beyond the components capabilities to provide a “safety factor” in the device.
As more pressure is being brought to the forefront on energy efficiency in
computing, manufacturers have been striving to match their power supplies more
closely to the components capabilities, or power parity.

One of the more power consuming components in most IT processing equipment are the
fans required to provide air for proper cooling internally within the equipment. These fans run continuously as long as the device is running. Both equipment and chip manufacturers have been making strides to better pair fan use with actual equipment needs. As chip development continues, heat tolerance is being increased.  Also, fans are being created which can be run in stages depending on the processing load of the equipment. This means that fans can run at lower speeds when processing is at a lower state, thus consuming less power.

Processing equipment developed within the last 3-5 years (depending on the manufacturer) is likely to be relatively energy efficient, anything older than that certainly
should be evaluated.

Consolidate and Virtualize

Another “low-hanging fruit” in many data centers is server consolidation and
virtualization. Typical utilization rates for non-virtualized servers is measured
between 5 and 10 percent of their total physical capacity, wasting hardware,
space, and electricity.  By moving to virtualized servers, data centers will be fully supported with less hardware, resulting in lower equipment costs, lower electrical consumption (thanks to reduced server power and cooling), and less physical space required to house the server farm.

It is important to remember that not all applications and servers are good candidates
for virtualization which adds complexity to the endeavor.

Along with consolidated server applications, associated storage for these systems is also becoming more consolidated as well. Storage Area Networks (SAN) and Network Attached Storage (NAS) solutions are becoming the norm in data center typologies. Virtualized tape systems are also replacing larger tape storage devices of the past. As these systems become
more regularized, they have also been increasing in density. This allows more storage in the same footprint with only marginal increases in energy consumption. The advent of solid state storage devices (SSD’s) will likely only create higher densities with lower overall power consumption in the future. Although these devices are not yet in production on central storage equipment, it will only be a matter of time before they are utilized.

The Bottom Line

A comprehensive efficiency strategy that targets IT processing equipment in addition to other tactics can substantially reduce energy consumption and net large savings. A facility-wide energy audit from an experienced partner will help to identify the areas where the most immediate impact can be achieved.

High Impact Measures to Boost Data Center Efficiency (Part 3)

Energy efficiency in electrical systems can be achieved through some measures to limit losses through devices among these components. Power parity (the amount of power put into a device equaling the amount of power provided to the device) provides for the most efficient use of power. Transformers and equipment which utilize transformers (such as UPS systems and PDU’s) tend to have some losses in efficiency due to the friction losses in the windings of these transformers. As equipment vendors apply more stringent manufacturing techniques to their products, improvements can be made to efficiencies of this type of equipment. UPS vendors now provide UPS systems which operate at a .95 or higher power factor. This means that there is only a 5% loss of power into the device
compared to power supplied by the device. It should be noted that these power factors are generally based on a load limit on the device no lower than around 30% of the rated maximum for the device, although some of the newer UPS systems can maintain their power factor down to as low as 20% of the rated maximum. As equipment is replaced due to changes in a system, end of life, or equipment failure, higher efficiency equipment should be specified and provided to improve on energy efficiency for these systems.

Measurement and Recording Data

We mentioned in part 1 of this series that in order to understand the consumption of power related to the data center, metering of these systems needs to be provided. Further, trending of this information is invaluable to understanding a baseline of energy use as well as the outcome of changes implemented to improve efficiency. The Power Usage Effectiveness (PUE) of the systems is an indicator of how efficient the data center operates. It is very important to understand where your data center ranks for PUE in order to know what measures should be taken to improve efficiency. This means that recording power usage at the main switchgear supporting both the electrical and mechanical equipment supplying the data center, and at the distribution side of the UPS systems distribution
(preferably at the 120/208 volt level at the PDU’s) is ideal to achieve the simplest means of calculating the PUE.

Lighting

Lighting systems have been moving towards more energy efficient components in recent years.  These systems have moved away from the use of incandescent and T12 luminaires to compact fluorescent and LED fixtures. ENERGY STAR has reported savings of 42% by switching from T12 fluorescent luminaires with magnetic ballasts to high efficiency T8 luminaires with electronic ballasts. It should be noted that oftentimes these higher efficiency luminaires actually produce higher lighting levels in addition to using less power. The more recent introduction of LED lighting luminaires, which can be retrofit into current fluorescent fixtures, is driving these efficiencies even higher.

Lighting Controls

Another energy savings measure which can be implemented in the data center is lighting controls. The notion of “lights out” data center operations refers to personnel not being normally stationed in the data center space. As operational controls of data processing applications become more network driven, and remotely accessed, less time is required in the data center to perform these activities. As a result of this reduced time spent in the data center, lighting becomes less necessary to operate under non-manned periods. Lighting controls utilizing occupancy sensors as a means of controlling lighting offers a reasonable solution to taking control of shutting off the lights out of the personnel entering and using the space. However, occupancy sensors do not allow for continued presence in the space when personnel are out of sensory contact with a motion or occupancy sensor due to working within or at the lower portions of equipment racks. In order to better accommodate these specialized circumstances in the data center, a combination of occupancy/motion sensors in conjunction with card access systems allows for a highly effective and efficient lighting controls strategy.

The Bottom Line

Once the proper metering components are in place and baselines are established, it’s relatively simple to determine which electrical infrastructure equipment will benefit from an upgrade and what the payback for the investment will be.  Also, paying attention to lighting controls can improve energy efficiency in the data center.  No matter what the situation is in your data center, a facility-wide energy audit from an experienced partner will help to identify the areas where the most immediate impact can be achieved.

High Impact Measures to Boost Data Center Efficiency (Part 4)

Mechanical cooling, depending on the efficiencies of the systems being used, can consume as high as 50% of the total power used in a data center. Good engineering practice, equipment efficiencies, and solid operational understanding can all benefit in a lower cost of ownership and operations.

Mechanical Economization or “Free Cooling”

The advent of “green” data center practices has ushered in a heightened interest in reducing mechanical systems energy use. As part of these efforts and in conjunction with data center design “best practices”, a means of mechanical economization or “free cooling” has become a design standard rather than a luxury.

Mechanical economization utilizes the ambient temperature of the local climate to provide an alternative means of heat rejection from standard mechanical systems. Two means of creating this ambient usage are through waterside or airside systems:

Waterside economization utilizes a liquid medium which is run through an outdoor series of coils to be cooled to a lower temperature.  If the ambient cooling meets the necessary set point required for the supply water temperature, the chiller barrel never needs to run, thus greatly reducing the power required for the chiller. During periods where the ambient temperature are not at levels to provide 100% economization, partial “free cooling” can still be provided reducing the overall power needs of the chiller, while still providing some mechanical cooling to reduce the return water to a proper supply temperature.

Airside economization utilizes an air exchange through either a cross stream configuration (which mixes return air with outside air passing through a filter media to help create the supply air stream) or a heat wheel (also known as an enthalpy wheel which nearly eliminates outside air mixing, but typically requires a much larger footprint). This system essentially eliminates the water system medium requirements. These systems can
outperform waterside economization in colder climates.

Mechanical Systems Controls and Monitoring

With mechanical systems improving their efficiencies through the equipment improvements and added systems designs noted above, controls and monitoring of these systems becomes more critical in order to maintain these efficient operations. CRAC unit manufacturers have added better controls for these units that allow for a more systematic approach for data center HVAC concerns. Units now communicate with one another throughout the data center and share individual operating conditions to assure a more singular response to the general room conditions.

Monitoring of these systems and trending data also benefit operations and maintenance personnel associated with the data center to better understand the effects of things like economization, maintenance, and other conditions which may affect the data center mechanical systems.

Motors and Drives

Because of reliability requirements in the data center, oftentimes mechanical systems are running at 50% or less their rated capacities during normal operation. This allows for failover scenarios to provide design loads even when a component in the system is not in operation. To further hamper efficient operation, most data center loads are typically below their maximum design capacities in order to plan for growth in the space.

In order to help alleviate the power consumption on equipment running at lower loads and help this equipment maintain better efficiency (as well as better life expectancy), the use of Variable Frequency Drives (VFDs) has provided a simple solution to allow better performance at lower loading while also reducing power consumption. A VFD is an electrical controlling device for motors varying the frequency to consume less power at
lower speeds when loads are not at their rated capacities. A motor can consume as low as 25% of the power required at 60% of the speed compared to 100% loading and speed. Additional benefits include reduced wear at start up and reduced over all motor wear by running it at lower rates than at its single speed maximum.

VFDs can be used in chillers, pumps, and cooling towers of a central system. VFDs can also be used on the air handler systems in the data center as well. VFDs can further provide more recordable information about power consumption for mechanical equipment. The recent improvements in the design technology of the new Variable Frequency Drives (VFDs) over the past few years have been substantial.  The operation of HVAC equipment, especially the pumps and CRAC Unit Fans, at reduced speed can produce cost saving of almost 20 percent.