The use of virtualization technology has been increasing in the IT industry to consolidate servers and reduce power consumption significantly. Virtualized commodity servers are scaled out in the data center and increase the demand for bandwidth between servers. Therefore, a high performance switch is required. The shared-memory switch is the best performance/cost switch architecture, but it is challenging to satisfy the requirements on the memory bandwidth in a high speed network. In addition, it is challenging to handle variable-length frames in Ethernet. This chapter describes the main challenges in Ethernet switch designs and then energy-aware switch designs, including switch architecture and high speed IO interface. As implementation examples, this chapter also describes a single-chip switch Large Scale Integration (LSI) embedded with high-speed IO interfaces and 10-Gigabit Ethernet (10GbE) switch blade equipped with the switch LSI. The switch blade delivers 100% more performance per watt than other 10GbE switch blades in the industry.
TopBackground
The ideal network is one that directly connects computing resources with unlimited bandwidth and no latency. To approach this ideal, high-speed serial interconnects have become more common in communications environments, and, as a result, the role that switches play in these environments has become more important.
The shared-memory switch is the best performance/cost switch architecture (Chao & Liu, 2007). Memory has the best buffer utilization because it is shared by all inputs and outputs. In addition, delay performance is also the best because of no head-of-line blocking. On the other hand, the shared memory must have sufficient bandwidth to accept packets from all input ports and write packets to all output ports at the same time. In other words, the shared memory for a switch with N ports at a line rate R must have a bandwidth of 2*N*R at least. For example, the shared memory for a switch with 20ports at a line rate 10Gb/s must have a bandwidth of 400Gb/s. If the memory access cycle is 10ns, the required data path is 4000bit wide which is not realistic. Several architectures have been proposed to satisfy the requirements on the memory bandwidth by using multiple shared-memory modules in parallel.
In the Space-Time-Space (STS)-type shared-memory switch, separate memories are shared among all input and output ports via switches (Oshima, Yamanaka, Saito, Yamada, Kohama, Kondoh, & Matsuda, 1992). Cells are written to the least occupied shared-memory first and the most occupied shared-memory last. This requires searching for the least occupied shared-memory. It may occur that two or more cells are read from the same shared-memory for different output ports. Therefore, this requires some kind of internal speedup to increase switch’s throughput.
In the Parallel Shared Memory (PSM) switch, separate memories are also shared among all input and output ports via switches (Iyer, Zhang, & McKeown, 2002). Cells are written to one of shared-memories by analyzing “pigeon holes.” Each time slot, up to N packets or pigeons arrive which must immediately be placed in to a shared-memory or pigeon hole. Likewise, each time slot up to N packets or pigeons depart. Cells are written to one of shared-memories so that the PSM switch does not allow a pigeon to enter a pigeonhole while another one is departing. It is challenging to determine the shared-memories in which arriving packets are placed.