SAS is the evolutionary successor to the Parallel SCSI interface. Like Serial ATA (SATA), Fibre Channel, and other serial technologies for storage, SAS was originally envisioned only as a point-to-point drive connection mechanism, but it has become much more. In its simplest configuration, SAS provides the physical connection between the host controller and some targets.

As the standard evolves, it becomes apparent that OEMs will need more robust extended connectivity fabrics to support large storage topologies. Thus, the concept of the expander was born. Like Fibre Channel switches, expanders provide a fabric for connecting multiple devices to a SAS domain, such as host controllers (initiators), hard drives (targets), and other expanders.

Large topologies of up to 16,384 devices in a single domain can be built with extended cascading and different connection routing mechanisms including direct, subtractive and table routing. Figure 2 illustrates a large SAS topology using expanders.

Extender Type

The two types of SAS expanders defined by the specification include edge expanders and fan-out expanders. Each type provides the connection mechanism needed to connect multiple targets to a single host or multiple hosts (one connection at a time). Both have common and unique properties, the biggest difference being whether a given type can be used in a cascading fashion. While fan-out expanders can be cascaded, edge expanders have limited cascading capabilities.

In general, edge extenders are best for designs where cost is an important consideration and storage scalability requirements are constrained. With the inherent expander building blocks of SAS, the next step in defining SAS as a fabric is a logical advance in the technology.

The composition of network interconnections or structures

Loosely defined, a fabric is a pathway on a computing, network, or storage device that provides chip-to-chip, adapter-to-adapter, or device-to-device connections for transferring information in a computing, network, or storage system/subsystem. Essentially, a Fabric is a switch or cooperative switching facility, much like an extender. A structure can be seen as a network and vice versa, at least a finite network.

SAS may become a fabric; however, as a point-to-point protocol, it was not originally intended to be a storage networking technology. SAS will likely need the following new capabilities to function as fabric or interconnect technology.

connection-oriented transport

SAS is a connection-oriented protocol, which means that a connection must exist between two SAS devices for data transfer to occur. By improving the SAS protocol to support a connectionless but reliable transport scheme, the problems of low link utilization, poor long-distance transport performance, and SATA/STP host starvation/locking can be avoided.

Enhanced physical connection

Unlike its network-enabled counterpart Fibre Channel, SAS does not currently have an optical interface. For most data center interconnects (rack-to-rack, rack-to-rack, or box-to-box), standard four-width SAS cables are more than sufficient. To support longer distances, an optical interface for SAS and its unusual out-of-bounds signaling must be defined.

Routing and Address Virtualization

The routing fabric in SAS was originally designed with direct connections and limited topology size in mind. Today’s top-level (fan-out) expanders require complete knowledge of the domains they connect to, limiting the effective size of the storage system. By adding route summarization, any extender in the domain does not need to maintain knowledge of the entire domain, allowing arbitrarily large sized topologies to be built.

Likewise, efficient routing requires an address virtualization scheme. Each SAS device has a set of hard-coded addresses (SAS addresses) that identify the device to the rest of the system. These addresses are either programmed at the factory or assigned by firmware at system startup. A mechanism must be established to allow the OEM to remap these physical addresses to more logical addresses. The Address Resolution Protocol will provide the basis for mapping hardware addresses to virtual addresses.

Smart Extender

Today, a SAS expander is essentially a circuit switch with a large amount of backing logic used to establish connectivity between SAS initiators and SAS targets. Most implementations are based on a cut-through type architecture, which means that it does not provide any buffering of frames with routing protocol data units (PDUs). Since expanders are the logical basis for making SAS fabric switches, the types of architectures described in the following discussion could allow SAS expander technology to evolve into the fabric of choice in the future.

Build the switch

Fabric switches come in many different sizes and flavors. Loosely defined, fabric switches are traffic directors that route PDUs from input ports to output ports according to some combination of criteria. The switch must also resolve any contention due to PDUs arriving at the common output port at the same time.

Most switches are based on one of several internal architectures: shared memory, shared bus (also known as shared medium), crosspoint matrix, and ring. Despite all the similarities, the underlying architectures are mainly differentiated from each other based on their buffer (queue) serving strategies.

Regardless of the architecture, a SAS switch such as an expander will ultimately contain the following key elements shown in Figure 3.

connection manager

Map the target SAS address in the connect request to the target PHY using direct, subtractive, or table routing

Arbitrates and allocates or rejects path resources for connection requests for arbitration and path recovery according to SAS rules

Configure the connection router

connect router

Route signals between PHY pairs (initiator and target) configured by the connection manager

Provides the routing resources needed to support the connection

broadcast processor

Route topology messages to appropriate devices (e.g. topology changes)

Device interface

Provides physical layer device interfaces for internal and external components

Arbitrate and route frames between PHYs

Provides physical and link layer connectivity for auxiliary I/O (eg, 10/100 Ethernet, general purpose I/O, etc.)

Key Innovations for SAS Advances

SAS can and will evolve by improving existing expander building blocks, adding optical interfaces, and enhancing transport protocols in key areas such as reliable, connectionless transport, route summarization, and address virtualization.

The future of storage and storage networking depends on the evolution of SAN and network-attached storage architectures, distributed models of storage, and advances in transparent protocol communication technologies. As with any new technology development, be it revolutionary or evolutionary, one size will never fit all. Complementary technologies will target different market segments, and the appropriate solution will vary by application, connectivity requirements, scalability, performance and price sensitivity. Only time will tell what this means for the future of SAS as a storage networking technology.

Reviewing Editor: Guo Ting

Leave a Reply

Your email address will not be published.