PCI Express 3.0 Introduction Opens Up Last Remaining Server I/O Bottleneck

While Sandy Bridge is the next generation architecture with PCI Express 3.0 for desktop PCs, Romley, is Intel’s Xeon (server) platform version. The introduction of PCI Express 3.0 addresses an important bottleneck in network traffic growth. Enterprise networking traffic often starts at the server, and with new servers moving from two or four to eight to twelve cores (with 50 cores on the horizon) and 400Gbytes of DRAM, higher speed interconnects are sorely needed, especially in light of trends to use virtualization to increase server utilization from 20 to 90%. PCI Express 2.0, or the Gen2 version, is the backbone bus for Intel/AMD-based servers and has been a severe I/O speed-limiting factor for many switch/router systems currently shipping. At IDF, Intel announced the Romley platform that interfaces the CPU to the PCI Express 3.0 bus and will be shipping in 2011 with data center servers and HPCs in the market early 2012.

The four-year-old PCI Express 2.0, at eight lanes and 5 Gbps per lane, had an affective throughput of about 27 Gbps with all the overhead and 8B/10B encoding delays. The new PCI Express 3.0 offers 8 Gbps per lane and the capability to support 16 lanes. Additionally, the new 128B/130B encoding with scrambling incurs very little overhead and offers features such as Quick Path, an express train that directly connects to the CPU and I/O bus and bypasses much of the bus arbitration delays; this doubles the effective throughput over PCI Express 2.0 and opens up the last remaining bottlenecks in servers in connection to the switch/router infrastructure. The 8Gbps transfer rate with the new encoding scheme is equivalent to a 10Gbps link using 8B/10B encoding.

While PCI Express 2.0 enabled dual 10GigE ports, the new PCI Express 3.0 will enable four-port 10GbE controllers, triple 4x10G QSFP, and even a 12x10G CXP port supporting 40/100GbE transceivers and AOCs for HPCs. This will enable Romley platforms to be used not only in servers but in switch, router, and storage platforms as well.

August 1, 2011

Continue