Data centres continue to grow in size, number and consumption; and as they do, approaches to their design have evolved. Low energy operation has become a design requirement, and the focus has shifted away from highly redundant mechanical and electrical infrastructure topologies to include both criteria. Many new data centre designs have a target PUE of 1.2 or below, whereas legacy facilities might achieve of PUE of 2.0 or above, and the industry has become more aware of its environmental impact. With raised temperatures and improved air management, free cooling has become possible across the globe, increasing the potential for zero refrigeration.
This case study looks at the efforts made by a global financial services firm in their legacy facility to reduce PUE from 2.29 to 1.49, and their annual electricity bill by $1.7M.
The legacy data centre is located in Greater London and has 2N systems. It is run by a financial institution and has 1,800kW of IT load in 3,500m2 data hall area, cooled by 53 CRAH units. The largest energy consumption was from the cooling system.
A six-stage energy improvement programme was used:
1. Energy assessment and data hall air temperature survey – identify improvement opportunities.
The operations team was trained to conduct the energy assessment themselves. First, the existing PUE of 2.29 was established, which contributed to a $4.7M annual electricity bill. Second, the air management was quantified and areas for potential improvement were established.
In this case 80% of the CRAH air was bypassing the IT equipment, returning directly back to the CRAHs, and 20% of server intake air was recirculated warm air from the server exhaust. Energy was being wasted moving 80% of the air with no cooling effect, and all servers had intake temperatures in the upper limit of the ASHRAE recommended range.
2. Implementation of air management improvements.
The following air management improvements were made, with return on investment of less than a year:
- Blanking plates installed to cabinets.
- Gaps in raised floor were sealed – cable cut-outs and gaps around PDU bases.
- Floor grilles were moved to where they were needed.
- Semi-cold aisle containment was installed throughout, with flame-retardant curtains fitted above racks to separate hot and cold air streams.
3. Reduced CRAH fan speeds and change to control on supply air.
With air now properly managed in the facility, it was possible to reduce CRAH fans speeds from 100% to 60%. Changing the CRAH unit control strategy from return to supply air control also helped to maintain air supplied at the server inlet to within a narrow range (within 1°C).
4. Increase in data hall air and chilled water set points.
Cooling unit set points were then changed, a degree at a time, from 24°C on return air to 22°C on supply air, with an associated increase in chilled water temperature set points from 6/12°C to 17/24°C
These temperature changes resulted in an increased COP of the chilled water systems and chiller delta T, resulting in $1.0M annual savings in electricity consumption.
Further changes included:
- Flow rate of outdoor air ventilation was minimised to pressurise the data hall, rather than provide a certain value of ventilation (not required by the design).
- Disabling of extract fans as design required pressurisation rather than ventilation.
- Widening of minimum humidity control in line with ASHRAE recommendations – from 50% RH to 5.5°C dew point control.
- Disabling of dehumidification due to limited hours of operation above 15°C dew point (ASHRAE maximum recommended).
These changes resulted in $0.3M annual savings in electricity consumption.
5. Feasibility of adding free cooling circuits to existing chilled water system.
The increased operating temperatures opened up the opportunity for free cooling operation. Several options were modelled, including cooling towers, and a dry cooler solution was chosen. 4 dry coolers were installed with both CRAH cooling coils operating simultaneously to leverage the additional heat exchange area. When both coils are used, the approach between leaving air and entering water temperature reduces to approximately half. Therefore, to achieve the same leaving air temperature, the chilled water temperature can be increased (by roughly half the approach). This results in a higher chiller COP and more hours of free cooling.
Dry cooler concept
Return water from the CRAH units passes through the additional circuit, which lowers the entering water to the chillers, reducing (sometimes eliminating) the chiller load. The dry coolers were added in series to allow additional hours of free cooling operation with partial free cooling. Connecting the dry coolers to the return section of the chilled water circuit meant little modification was required to the chiller controls. Also, connecting the dry coolers on the secondary side means they are supplied with warmer return temperatures, compared with on the primary circuit where there is some bypass flow, which again increases the number of possible free cooling hours.
Engineering analysis was used to model the predicted behaviour of the cooling systems. The energy model predicted 100% compressor free, free cooling operation for 63% of the year annual plus an additional 33% of partial free cooling (i.e. total or partial free cooling for 96% or the year).
6. Installation of free cooling modification using dry coolers.
The dry coolers were installed, and are now running. These changes resulted in an additional $0.4M annual saving in electricity consumption with a return on investment of less than 3 years.
The savings described in this article have reduced the annual electricity bill by $1.7M from $4.7M to $3.0M. Further savings were made on top of those described, by optimising the UPS system and lighting loads. The case study shows that these savings are possible, not only on legacy sites, but also in high availability environments.
Bearing in mind the criticality and live nature of the facility, changes were implemented incrementally and their impact monitored closely and continually, to ensure the same high level of reliability was maintained. Because of this, the savings were achieved over a period of time. Rather than stopping at the end of the programme, however, the operator has continued to increase data hall supply temperatures and chilled water temperatures leading to further savings, and has instigated similar programmes throughout their global portfolio.
Key to the success of the programme was buy-in from the operational team, who with training drove the optimisation process. The energy saving programme included a workshop element, which provided a common language for all teams to work together towards a mutual goal of saving energy, enabling them to share cross-discipline knowledge