The Supercomputer Supernet: A Scalable Distributed Terabit Network

Abstract

Conventional supercomputer interconnection networks consist of crossbar modules, which are connected by point-to-point copper or fiber links to create distributed mesh topologies (e.g., CP*, Nectar). This type of “physical networking” topology creates cable layout problems, dealing with bundles of cables/fibers between various pairs of modules. It also introduces several routing hops, increasing the probability of interference between connections and making it difficult to guarantee quality of service to real time applications. We describe a new network called the Supercomputer Supernet (SSN) that attempts to overcome these problems by replacing the point-to-point links with a fiber optic interconnect system. The novel scheme employs asynchronous pipeline crossbar switches (APCS) used in parallel supercomputers to interconnect multi-channel wavelength division multiplexed (WDM) fiber optic links to an optical star (or tree) “physical” topology. WDM will be used to subdivide the very large fiber bandwidth into several channels each of Gb/s bandwidth. WDM channels (supporting also time division multiplexing) will be established between modules, thus defining a dense “virtual” interconnection topology, which is dynamically reconfigurable, responding to changing traffic patterns. A pool of channels will be set aside for direct, end-to-end connections between crossbars, providing circuit-switched service for real-time traffic applications.