Pavel Thomas, Director EMEA sales, storage products, Avago Technologies

Given the colossal avalanche of data, how to handle such volume profitably? And how to maintain or improve the performance of data storage? By eliminating bottlenecks. Here are some tips.

Elimination of data storage bottlenecks
The data deluge is not ready to end, especially with the surge of demanding applications such as the value and profitability analysis programs, social media, video distribution and grid computing to name a few examples. This data growth presents many challenges for system architects. How to handle such volume profitably data? And perhaps most important: how to maintain or improve the performance of data storage?

In fact, there are very few who are not affected by the data deluge. Even home users generate more video and high quality digital pictures which occupy considerable space on PCs and smartphones. This is one of the reasons why social networks such as Facebook are generating half a petabyte of data every single day (according to the notes of Facebook engineers in 2012).

And this increase in the quantity of data is accompanied by a longer access time. This article describes the ways in which the third-generation SAS technology eliminates the various data storage bottlenecks.

Understanding where bottlenecks occur
When designing data storage systems, it is essential to understand where bottlenecks occur. Note also that these bottlenecks are changing with every new technology, along the data storage path. The three most important factors that affect performance storage are the server-side PCIe® bus (Peripheral Component Interconnect Express), the solution deployed by SAS HBAs (Host Bus Adapters) and expansion dispatchers whose interface can be either SAS or SATA (Serial Advanced Technology Attachment).

SAS technology is now in its third generation, and performance doubled with each generation, the 3 Gb/s of the beginning has evolved to 6 Gb/s, and nowadays to 12 Gb/s. The SAS System, like PCIe, uses channels and high-performance storage systems typically combine multiple SAS ways to support the most important data rates. The HDDs (Hard Disk Drives) and SSD (Solid State Disks) are capable of supporting either SAS or SATA storage interfaces.

Storage systems performance guidelines
When designing a high-performance storage system, it is necessary to understand the performance limits of each. Typical yields of different types and generations of the three main elements discussed in this article are summarized in Table 1.

Table 2 summarizes some typical configurations and shows the location of the bottleneck when the system has a “suitable amount” of hard disks (slower elements of the system). As reported, to support more disks (for purposes of capacity) more PCIe, and / or more SAS channels should be used. If you look at it in a different way, in systems that have small numbers of records, the low cumulative return becomes the bottleneck, and it is not necessary to “overprovision” design by adding more recent technologies and / or more SAS lanes into the mix. The disks listed in Table 2 were all a 6 Gb/s interface and achieve 230 MB/s and 550 MB/s for 15 000 rpm HDDs and SSDs, respectively.

Note that in Table 2, all disks are supposed to work to their peak efficiency simultaneously and that does not always happen. It is also important to note that the IOPS are often more critical than raw throughput in many current applications, depending on the circumstances. It is for these reasons that each configuration is normally able to support more disks.

Evolving to 12 Gb/s SAS technology
The main problem when it comes to move to third-generation 12 Gb/s SAS technology, is well known: it is that of investment protection. Most organizations have invested heavily in SAS disks and want to preserve that investment as they pass the SAS 12 Gb/s barrier. The difficulty being that third-generation SAS standard maintains backward compatibility by adopting the SAS slower system data rate.

In small point to point configurations, this is not always a problem, because migration would require upgrading the initiator and the target. However, in most organizations, this type of configurations are quite rare. Therefore, the performance limit at the slowest data rate at the system level does not allow these organizations without point to point configuration of obtaining the performance of 12 Gb/s since all disks are not SATA-III compliant.

Fortunately, it is possible to overcome these limitations, and this requires a slightly better understanding of how SAS expanders work. A SAS expander enables one or more initiators to communicate simultaneously with multiple targets as shown in Figure 1. The expanders give the SAS its remarkable evolution faculties, and as each is able to support multiple disks they also allow to combine the performance of these drives.

Figure 1 shows 48-port expanders connected to two redundant RAID controllers and a shared disk array. Arrays of additional drives can be daisy chained, and some SAS expanders can support up to 2,000 disks in such configurations.

According to the third-generation SAS standard if a SAS or SATA drive operates at
6Gb/s with RAID controllers also working at 6Gb / s. But if the bandwidth of two 6 Gb/s disks must be combined into a single 12 Gb/s channel, how many channels can be combined with PCIe?

This combination of bandwidth and performance immediately boost IOPS while protecting the investment in first and second generation SAS storage systems. This allows to introduce 12 Gb/s SAS systems in an existing 6 Gb/s storage array in order to immediately double the overall system performance. In fact, with the combination of bandwidth, it is not even necessary to use last generation 12 Gb/s SAS drives to achieve 12 Gb/s SAS performance at the system level.

The combination of bandwidth works by using a buffer per 12 Gb/s port, to allow what appears to be a communication between 12 Gb/s SAS initiators and any speed (3 Gb/s or 6 Gb/s) SAS or SATA targets. Stacking technology is proprietary, but it operates entirely within the expander, allowing to standardize all interfaces. In this way, a SAS port 12 Gb/s SAS drive connected to a 6 Gb/s SAS port would operate as a second-generation backward compatible 6Gb/s, but the whole system would work at the third generation 12 Gb/s data rate.

Conclusion
Data storage systems bottlenecks change depending on the generations of technologies connected end to end. With the advent of third-generation PCIe, for example, the second generation SAS technology has become the new bottleneck in data storage. The third-generation SAS technology now gets the best performance out of third-generation PCIe, making the new PCIe interface bottleneck on 12 Gb/s SAS systems.