Boundless Freedom for Edge Servers
Article By : Dan Demers, congatec
The integration of Intel Xeon D processors on COM-HPC Server-on-Modules allows edge server installations to break free from the tight thermal constraints of air-conditioned server rooms.
Edge servers process data at the edge of communication networks instead of in central clouds. This enables interaction with clients of all kinds without delay or in real time but presents manufacturers of server, networking and storage technologies with major challenges. Until now, they used to develop standardized rack solutions for their systems, with active ventilation concepts and powerful air-conditioning technology to control the thermal management of the racks and the air-conditioning in the server rooms. However, such an approach is often no longer a good fit for the edge server technology of today.
The American Society of Heating, Refrigerating and Air-Conditioning Engineers, or ASHRAE for short, has looked thoroughly at the question of how to best install edge server performance in harsh environments. So, from the perspective of the associated heating, cooling, ventilation and air-conditioning companies, there are already quite plausible recommendations on how to design edge data centers with high-performance air-conditioning and the best possible insulation to protect them from heat and cold.
Freeing edge servers from the shackles of air-conditioning
【Webinar】Achieving High Defect Coverage for Safety Critical and High Reliability Designs
However, ASHRAE proposes a maximum permissible temperature fluctuation of 20°C within one hour and a maximum of 5°C in 15 minutes for edge data centers. This requires complex air-conditioning technology and is therefore very difficult to implement. But not only that; adhering to these guidelines is near impossible, especially during maintenance work in edge data centers that are smaller than a telephone booth, because such solutions must be opened for maintenance at any ambient temperature. It is simply not possible to slip into such systems via a climate control chamber and quickly close the door again before carrying out maintenance work in the fully air-conditioned edge server room.
Edge servers and data centers operating in harsh environments therefore need system designs that can cope with greater temperature fluctuations and a much wider temperature range than the 0‑40°C that is common for indoor IT (see Figure 1). In industrial environments, embedded system designs can be exposed to ambient temperatures ranging from an arctic -40°C to a searing hot 85°C. Which means each component must be hardened.
Figure 1: Standard motherboard designs usually only support standard interfaces onboard that are executed at the rear of the board (rear I/Os). As this doesn’t take industrial requirements into account, their suitability as edge servers for the Internet of Things is limited. And as a rule, they are neither designed for the extended temperature range of -40°C to +85°C nor do they guarantee long-term availability of 7 to 15 years. With Server-on-Modules, however, it is possible to use the mechanics of these form factors and design a carrier board that executes the desired interfaces where they are required. (Source: congatec)
Rugged designs reduce air-conditioning costs
A particularly challenging decision point in the design of edge server, networking and storage technologies is the choice of processor technology. The decision that stands and falls with this choice is whether to follow the ASHRAE recommendations and invest massively in air-conditioning technology and insulation, which comes with high investment and operating costs for secondary energy. Or whether to develop systems that don’t need any of that because they work reliably even in extreme temperatures and can therefore be implemented in harsh environments much more cheaply – from factory installations to outdoor communications, video surveillance and other critical infrastructure equipment to servers in mobile systems ranging from trains and airplanes to self-driving shuttle buses in smart cities.
Thanks to the new Intel Xeon D processors, there is now a very powerful server technology that is qualified for use in extreme temperature ranges from -40°C to 85°C. Even ultra high-performance server designs are no longer constrained by the tight thermal restrictions of air-conditioned server rooms. Ultimately, they can be deployed wherever latency-free massive data throughput is required at the edge of the Internet of Things and in Industry 4.0 factories (see Table 1 and 2).
Table 1: Several variants of the Intel Xeon D 2700 HCC processor support the extended temperature range. They are available on COM-HPC Server Size E Modules.
Table 2: Also, the Intel Xeon D 1700 HCC processors support extreme conditions. They are integrated on COM-HPC Server Size D and COM Express Type 7 modules.
High system design demands
However, a server processor alone does not make a rugged edge server. Meeting the system design demands for harsh environments also requires extensive know-how. Every single component used must be qualified for this environment, and special requirements also apply for the circuit board and board design. Examples are special coatings that protect against condensation water and other environmental influences, or a high level of protection against extraneous electromagnetic and high-frequency signals that could hamper device performance.
Developers of embedded computing technologies have decades of experience in the design of such systems. They have long been integrating standard PC technologies such as Intel Core processors into embedded systems in a way that is suitable for industrial use. They know the requirements and certification standards of a wide range of industries inside out; and they are used to designing their systems for long-term availability in order to meet industry requirements and to be able to supply OEM solutions with identical board configurations for 7, 10 or 15 years. They also know that industrial applications differ significantly from standard system designs for the office environment, because industrial applications always require a greater or lesser degree of customization, which makes modular designs that deploy Computer-on-Modules the ideal way to develop boards. They have also learned that standardization is key, which is why they helped to create globally recognized standards for such modules.
Reaching the goal faster with standards
Figure 2: congatec offers the Intel Xeon D processor on COM-HPC Server Size E and Size D as well as COM Express Type 7. (Source: congatec)
With the new COM-HPC Server specification in place, and the launch of the Intel Xeon D processors, this bundled expertise has now been transposed to industrial edge server designs. For the first time, developers have access to real products (image 2). The advantage of these new standardized COM-HPC Server-on-Modules is that developers can integrate them into their custom carrier boards as application-ready embedded computing logic. This means they don’t have to concern themselves with basic processor technology issues, but only deal with the application-specific positioning of the board components and execute the interfaces at the right place on the carrier board (image 3).
To this end, the PICMG standardization committee recently released the COM-HPC Carrier Design Guide. It provides essential guidelines for building interoperable and scalable customer-specific embedded computing platforms based on the new standard and also makes it easy for developers to understand the logic behind the standard.Special focus is on communication interfaces and how to avoid pitfalls in the challenging design of high-speed serial communications: from PCIe Gen 4 and 5 to USB 3.2 Gen 2 and USB 4 with Thunderbolt on USB-C to 100 Gigabit Ethernet, and also including the management of sideband signals for 10G / 25G / 40G / 100G Ethernet KR interfaces, which in COM-HPC must be deserialized on the carrier board. Supported by all leading embedded computing vendors this set of PICMG standards offers engineers the benefits of best in class design security.
Figure 3: The modular Server-on-Module approach makes it easy to develop dedicated edge servers with application-specific interface layouts using custom-designed carrier boards. (Source: congatec).
Accelerating edge server workloads
The first COM-HPC Server-on-Modules in Size E and Size D with BGA-mounted Intel Xeon D processors (code name Ice Lake D) available from vendors like congatec impress not just with support for the extended temperature range from -40°C to 85°C. They also break many of the previous bottlenecks caused by edge server restrictions and will significantly accelerate the coming generation of real-time micro server workloads in harsh environments and extended temperature ranges. Improvements include up to 20 cores, up to 1TB memory on up to 8 DRAM sockets at 2933MT/s, up to 47 PCIe lanes per module in total and 32 PCIe Gen 4 lanes with double throughput per lane, and up to 100 GbE connectivity and TCC/TSN support with optimized power consumption thanks to 10nm manufacturing. Video storage and analytics servers also benefit from integrated Intel AVX-512, VNNI and OpenVINO support for AI-based data analytics (see Table 3).
Table 3: Intel Xeon D processor based COM Express Type 7 and COM-HPC Server Size D modules don’t just differ in size but also in pinout. (Source: congatec)
A milestone for real-time edge server designs
In fact, the market launch of the Ice Lake D based COM-HPC Server-on-Modules sets a triple milestone: First, because extended temperature range support means Intel Xeon D Server-on-Modules are no longer limited to standard industrial applications but also target outdoor and automotive environments. Second, the world’s first COM-HPC Server-on-Modules increase the number of available cores to 20 for the first time; with up to 8 DRAM sockets, this provides massively more memory bandwidth than Server-on-Modules based on other PICMG specifications. Third, these new server modules are real-time capable, both in terms of processor cores and TCC/TSN-enabled real-time Ethernet, which is essential for digitalized IIoT and Industry 4.0 projects.
To be able to implement server balancing and server consolidation services for deterministic real-time edge server installations, where diverse real-time applications operate independently of each other on a single edge server, it is helpful if the platforms support real-time-capable virtual machines, as the RTS Hypervisor from Real-Time Systems does, for example (see image 4). This allows Industry 4.0 factories to host heterogeneous real-time applications on a single server platform at the edge of their private 5G networks, and to allocate exclusive system resources to the individual processes. Designers should therefore look for Server-on-Modules which are pre-qualified for such services.
Figure 4: Up to 20 cores deliver a massive choice of secure options for a wide variety of real-time applications when using virtualization technology from Real-Time Systems. (Source: congatec)
COM-HPC Server modules further impress with a comprehensive server-grade feature set: For mission-critical designs, they offer powerful hardware security functions such as Intel Boot Guard, Intel Total Memory Encryption – Multi-Tenant (Intel TME-MT) and Intel Software Guard Extensions (Intel SGX). For best RAS capabilities, the processor modules integrate the Intel ME Manageability Engine and support remote hardware management features such as IPMI and redfish. In fact, there is another PICMG specification that ensures the interoperability of such implementations, and the training program of the congatec academy also covers this aspect.
About the Author
Dan Demers holds a B.B.S degree in International Business from Grand Valley State University, Grand Rapids, Michigan and an M.B.A. from Ashford University, Clinton, Iowa.Mr. Demers has over 22 years of experience in embedded computing having worked with Fortune 500 companies in the Industrial, Medical, and Communications markets.
