Storage – What is the Right Choice for You?

This article provides an overview of common storage systems and their pros and cons. To help you make the right decision.

• By Christoph Bast
• Nov 12, 2015

Everyone talks about it; everyone knows it: The requirements for video surveillance are continually on the rise. Even more cameras, higher resolutions and longer retention times are increasing the demand for storage systems. When you need a new storage system, it is important to first be clear about your specific requirements. Put another way: You must define your needs in terms of reliability, capacity, cost and performance. Of course these requirements are interdependent, making the decision that much more difficult. This article provides an overview of common storage systems and their pros and cons. To help you make the right decision.

Reliability = Availability

Reliability is a great virtue. For video systems, as well. In this case, it can even be defined: Reliability means availability. How much you need depends on the application. The following question may be able to help you assess your needs: How critical is it if individual sequences of images are missing in the recording? What would happen if one in ten incidents were not documented? Or one in a hundred? Of course, everyone would prefer absolute reliability and thus 100% availability. Technically speaking, this level is almost impossible to realize. And of course, more security costs more money. The level of availability is therefore always a compromise between requirements and the available budget.

But even here the situation is not black or white – the possibilities are vast! See for yourself:

The basis: RAID

RAID stands for Redundant Array of Independent Disks. It is a method of storing data on multiple individual disks that are combined into a single logical unit. A RAID controller controls the write and read operations, so that the entire system achieves a higher level of reliability or a higher data throughput than the individual physical storage medium itself. RAID is therefore more than just data backup. When one hard disk fails, system functionality remains intact (exception: RAID 0). After replacing the failed disk, the original state of the storage system is restored by a rebuild.

1.1      RAID – what is that exactly?

Depending on the objective, different RAID configurations, also called levels, are used. In all, there are about a dozen different combinations of configurations. Four types are used most often:

RAID 0 or striping

• High data throughput: Data is distributed across multiple disks with the aim to achieve a high data transfer rate. The bottleneck in storage access is the mechanical limitation of the hard disk (the speed of read/write unit). By controlling multiple disks in parallel, it is like their speeds are "added together".
• No data security: On the contrary, because when one disk fails, not only is all the data on that disk lost, the data on the other disks are at risk as well due to the fact that the information is distributed across them. If one disk fails, all data records are affected whose fragments were stored on this disk – in most cases this means the entire array of disks. Recovering the data records that were not affected is only possible using time consuming methods.
• Not a realistic option for video security: Due to the lack of reliability, RAID 0 is not recommended for use in video surveillance systems.

RAID 1 or mirroring

• Duplication of data: All data is mirrored, that is, it is stored twice.
• Maximum reliability: This method provides a very high level of reliability as well as high performance.
• High costs: Doubling the number of hard drives is usually quite expensive
• The net capacity is cut in half: Only half of the total capacity of all physically present hard drives is available for storing data.
• Special area of use: The method is particularly useful when the entire dataset must be removed for external use without interrupting system operation, for example for use as evidence.

RAID 5 and RAID 6 – the standards

• Reliability: User data and parity data (generated during data storage) are stored across all disks. Using the parity data, all data can be completely recovered, even if one disk fails (RAID 5). In a RAID 6 configuration, it is even possible for two hard drives to fail simultaneously, as in this case the parity data has itself been backed up by a duplicate.
• Continuous data access: Once a defective disk is replaced, the system automatically starts with the rebuild (recovery process) of the parity data so that once this procedure is complete the original level of reliability is again provided. During this period, it is still possible to access all user data.
• Limited costs: The net usable capacity for storing the user data depends on the selected RAID level: In RAID 5 it is n-1, for RAID 6 it is n-2, where n is the total number of disks. In a RAID system with a total of 16 hard drives, 2 disks are used to ensure RAID Level 6, leaving 14 hard drives to store the user data.
• Balance: Users for whom reliability, data throughput and cost efficiency are equally important often use RAID 5 or RAID 6.

Our advice: Since the performance of RAID systems depend on the proper interaction of hard disks, it is advisable to only use identical hard disk drives. Identical means: Same manufacturer, same capacity, same type number and, if possible, the same firmware version.

Another thing: In RAID systems we exclusively use so-called "enterprise" HDDs that are certified for this use. They usually are characterized by a longer service life and reliability than "desktop" drives.

1.2      Which RAID level should be used for video security?

Although RAID is a complex topic, this question can be answered simply: The trend is moving toward RAID level 6, in particular as hard drives continue to increase in capacity.

With double parity, RAID 6 provides protection against the simultaneous failure of two drives. This means that even when the system is performing a rebuild, the RAID array is still protected against the failure of another hard drive. The intense read and write operations during the rebuild in particular place increased demands on the drives. In addition, hard drive sizes continue to grow unabated, meaning the recovery process takes longer as well. The risk that a drive fails during a rebuild thus increases with the hard drive capacity. RAID 6 ensures the reliability of the system even during rebuilds – and at the price of only one additional disk!

An alternative is RAID 5 with an integrated "hot spare" disk: A reserve disc that is incorporated in the system but remains inoperative until one active disk fails. Only then is it activated by the RAID controller and the rebuild process starts immediately. The advantage: The "hot spare" disk hardly ages for as long as it is not in operation. The disadvantage: During the rebuild there is no security against failure of an additional disk. However, the rebuild time is reduced. This makes sense if a system cannot be easily accessed by the system administrator, and as a result it can take some time until a defective disk can be replaced. In principle, "hot spare" can also be integrated into a RAID 6 system.

Our advice: Be sure to enable Controller Messaging! Ensure that your administrator takes the message seriously and keep spare disks ready on site. The administrator can thus react quickly and minimize the risk of data loss.

The overall concept: Central or local

It is and remains a matter of priority: Do you need maximum storage capacity of hard disks, optimum data throughput or maximum reliability?  Not just the RAID level within a system counts – the selection of the storage concept and the connection with the data-generating devices also have their influence. In general, a distinction is made between network attached (central) and directly attached (local) storage. These are the differences:

1.3      iSCSI Storage – central storage over longer distances

iSCSI (Internet Small Computer System Interface) is a standard for storage networks based on IP technology, and it is used for centralized management of storage within a LAN. It is used in particular when multiple devices generate data that are stored on a shared storage system. It can be used to save space, for improved administration or to combine the (individually small) storage requirements of multiple units into a larger array. Assuming RAID 6 is used by default, the following variants exist:

For maximum storage capacity, all available disks (n) are combined to form a large storage system, a "volume group". The net capacity (RAID 6 = n-2 – see above) is then divided into individual partitions (Virtual Disks). Each partition contains a corresponding portion of the available storage capacity. The data generating computers (in our application DVRs or NVRs, i.e. recorders) now use "logical", "virtual" locations instead of local storage media. A potential bottleneck is the centralized storage management, which receives and distributes the data from all recorders – at a limited data throughput. The advantage, however, is that the RAID 6 reliability "costs" only 2 hard disks for the total capacity of all recorders.

Image:             image_1.jpg

Copyright:       © Geutebruck

Caption:          All recorders share a volume group with RAID 6. The maximum data throughput is shared between the devices. The access speed for each recorder is limited.

For even more data security, multiple volume groups within a RAID system are each configured with a RAID 6 system. Each volume group is a separate partition (Virtual Disk). Each recorder uses a separate volume group for its data. Each volume group is controlled separately. Benefits are even greater reliability and more efficient management by the RAID controller with a correspondingly higher data throughput per recorder. However, this concept "costs" significantly more hard drives – namely exactly twice the number of volume groups – that are not available for user data.

Image:             image_2.jpg

Copyright:       © Geutebruck

Caption:          Management of the RAID system is simplified. The performance of the individual units increases. Writing, reading and access are faster. When a hard drive fails within a volume group, only one recorder is affected. 

1.4      SAS Storage – the direct link

SAS (Serial Attached SCSI) is a storage system directly attached to the computer (also known as: Direct Attached Storage (DAS) or Server Attached Storage). There is a dedicated, direct cable connection between the computer and storage unit via a SCSI cable. The distance between the two is limited to a few meters. The overall performance of the RAID system is 100% available for the connected recorder, but only for this recorder. Each additional recorder needs its own storage system. The advantage is very high data throughput and therefore optimum disk access. This approach is recommended if the storage requirements for an individual recorder are high, for example when the images from a large number of high-resolution cameras need to be stored over a long period of time.

Our advice:
Use recorders with an integrated RAID system such as the G-Scope/8000 and expand the storage up to the capacity you need using SAS-connected JBODs (Just a Bunch Of Discs). They are cost-effective expansion units, which are controlled by the RAID controller of the recorder.

Image:             image_3.jpg

Copyright:       © Geutebruck

Caption:          A G-Scope/8000 with integrated RAID system can be expanded using JBODs to a database size of 256 TB.

Summary

A storage system is used to improve performance in at least one major aspect compared to operation of one or more individual hard drives. Depending on the configuration, a RAID system offers the following advantages:

• Improved reliability of data storage
• Guaranteed high availability 
• Improved data throughput rate