Distributed Storage Done Right

Debate revolves around scalability and cost effectiveness

Scalable system architectures have been the source of many debates within the security industry. A particular point of the debate revolves around distributed video storage and its ability to scale quickly, easily and cost effectively. As distributed storage continues to gain broad mainstream appeal, it’s worth revisiting the discussion.

Architectural Advantage

Modern networks are designed to scale with applications that follow a few-to-many streaming model—a small set of servers stream to many clients. Multicast is one example of such a network feature. But traditional IP video surveillance is inherently a many-to-few streaming application—many cameras stream to a few servers. Thus, IP video surveillance networks are plagued by bottlenecks not anticipated by today’s network infrastructure. By distributing primary storage and processing to the edge and allowing end users to intelligently stream on demand, distributed storage and processing systems adapt to the strengths of today’s network infrastructure rather than being limited by its constraints.

Tests have shown that distributed systems eliminate more than 90 percent of the network traffic of a traditional, centralized surveillance system because only the video that is being actively viewed is transmitted across a network—no video needs to be backhauled to central storage. If centralized archiving of important content is needed, then the backhaul can be scheduled to accommodate the load on the network. Additionally, customers can choose lower resolution and bitrate streams for live viewing while still recording full-resolution, high-quality video at the camera, further reducing bandwidth consumption without sacrificing the forensic value of evidence.

Finally, distributed systems eliminate the need to manage and archive video in a large, common repository—the greatest computing and storage resource bottleneck in a central architecture. The result is a system that is network and computer efficient, allowing systems to scale from a single camera to thousands of cameras with minimal infrastructure.

Market Adoption

There has been a significant increase in companies integrating with distributed storage providers over the past year. The mainstream support and acceptance of distributed storage by large VMS companies will ensure that the architectural benefits of distributed storage will be available to mainstream end users whether they require one camera or thousands.

Important Considerations When Evaluating Long-term Storage Options

  • Solid-State Disk (SSD) storage is an expensive solution for long-term storage. SSDs are thought of as an appealing option for long-term storage due to their perceived reliability and data throughputs. But not all flash storage technologies are equal in performance. Those that measure up to expectations are cost prohibitive for long-term storage applications.
  • HDD alone is not reliable. HDD storage is the current standard for long-term storage because it provides the best cost per gigabyte with theoretically sufficient reliability to meet customers’ needs. Spinning magnetic media storage technology has evolved significantly during the past decade and a half. However, as any experienced integrator or end user knows, hard drive reliability is a constant concern and an expensive source of frustration.

Unfortunately, HDD failures are simply a fact of life. Accounting for failure concerns often requires elements of redundancy and fault tolerance to be factored into the system architecture. Depending on the nature of the installation, more complex fault-tolerant storage architecture increases up-front costs and installation time. In large-camera-count installations, the cost of building in fault tolerance is a small percentage of the overall system costs and thus may not influence the storage decision.

However, in smaller installations where cost is a significant factor and decision driver, the end user who does not build a redundant storage system might be forced to suffer through downtime and lost video.

  • Hybrid storage delivers high reliability at a compelling price point. As a general rule of thumb, the lifetime of an HDD is correlated with the number of spinning hours of the disk. In
    most user-driven applications, disk access is often “bursty”
    and not continuous. In other words, even with heavy workloads,
    the expectation is that the disk is not spinning 24/7.
    Typical surveillance applications require continuous 24/7
    disk access, accelerating most vendor mean time between
    failures (MTBF) estimates.

A widely cited 2007 study by Carnegie Mellon University’s
Bianca Schroeder and Garth Gibson showed that even enterprise
HDDs don’t offer significantly longer MTBFs for continuous
access applications. The duo’s work also brought into
question some advantages of RAID 5.

There are hybrid storage solutions available that combine the
robustness of industrial SLC flash storage with the cost advantage
of HDDs, up to 750GB. The combination works by first
filling the flash storage with video. Once the flash storage is full,
it spins up the HDD storage, writes the stored video to the HDD
and then suspends the HDD.

More specifically, the capacity of the solid-state storage is
sized such that the HDD is spinning for a small fraction of every
hour. Because the HDD is suspended and not spinning for the
majority of the hours in a day, the disk usage model is consistent
with what manufacturers account for in their design. Thus,
in video surveillance applications, the MTBF can be four times
longer than using an HDD alone. Additionally, advanced analytics
control compression quality based on video content, reducing
the total volume of video both for continuous recording
and event recording. This further reduces HDD spin time and
contributes to increased MTBF.

Important Considerations When Evaluating Short-term Storage Options

There are numerous situations when an integrator, remote
monitoring company or end user will need short-term storage,
typically a few days to a week of storage. In some cases this is
because the cameras are being continuously monitored and any
evidence of loss or a crime is found within minutes or hours, let
alone days.

In other cases, the surveillance solution includes long-term central
storage, using distributed storage mainly as a fault-tolerance
mechanism protecting against network failures and downtime.
Many cameras and encoders offer SD cards as a solid-state solution
that provides reasonable storage capacities at moderate prices.

There are better solid-state storage options than consumer
SD cards. Flash memory actually comes in several variations
differing in speed, reliability and cost. SLC NAND flash memory
is the high end of the flash variants, providing the most reliable
storage available with a high cost to match. Each memory
cell in SLC NAND has 10 times longer lifetime than its cheaper
sibling, MLC NAND.

MLC NAND flash fills out the middle and lower end of
flash storage options with multiple variations within the MLC
category. MLC NAND flash in the mid-range provides a good
combination of speed and reliability without breaking the bank.
The larger the capacity of flash storage, the fewer times a single
memory cell is rewritten. Thus, theoretically, MLC flash storage
with 10 times the capacity of SLC flash will have about the
same reliability for sequential-write applications (such as storing
video). Unfortunately, when MLC flash storage capacities
go beyond a point, read/write latencies and errors also increase.
Thus, MLC NAND flash is a good solution for applications
requiring short-term storage. MLC NAND-based SSDs up to
256GB in capacity are now fairly common and have been shown
to have good reliability.

The lowest tier of MLC NAND flash is used in most consumer
SD cards. These types of low-quality flash storage are not
designed for applications that require gigabytes of data per hour
to be written 24 hours/day. Though an SD card slot is a common
choice on many surveillance cameras and encoders, reliability
will be a serious issue if the SD card experiences continuous
video writes. If installers or end users are not careful about their
choice of consumer or professional SD cards (those that use
higher-end MLC NAND), premature media failure is inevitable.

Edge storage creates scalable surveillance networks. As discussed,
one of the most reliable choices is a hybrid-storage option
that delivers high reliability at a compelling price point.
When considering long- or short-term storage options, make
sure you understand the different storage choices, what they deliver
in terms of reliability and capacity, and when you should
make trade-offs between cost and reliability.

This article originally appeared in the December 2012 issue of Security Today.


  • CISA Kicks Off 20th Anniversary of Cybersecurity Awareness Month

    CISA Kicks Off 20th Anniversary of Cybersecurity Awareness Month

    The Cybersecurity and Infrastructure Security Agency (CISA) recently announced the kickoff of the 20th Cybersecurity Awareness Month. Throughout October, CISA and the National Cybersecurity Alliance (NCA) will focus on ways to “Secure Our World” by educating the public on how to stay safe online. Read Now

  • Cybersecurity Awareness Month: Top Five Action Items to Elevate Your Data Security Posture Management and Secure Your Data

    October is Cybersecurity Awareness Month, and every year most tips for security hygiene and staying safe have not changed. We’ve seen them all – use strong passwords, deploy multi-factor authentication (MFA), be vigilant to spot phishing attacks, regularly update software and patch your systems. These are great recommended ongoing tips and are as relevant today as they’ve ever been. But times have changed and these best practices can no longer be the bare minimum. Read Now

  • Boosting Safety and Efficiency

    Boosting Safety and Efficiency

    In alignment with the state of Mississippi’s mission of “Empowering Mississippi citizens to stay connected and engaged with their government,” Salient's CompleteView VMS is being installed throughout more than 150 state boards, commissions and agencies in order to ensure safety for thousands of constituents who access state services daily. Read Now

  • Live From GSX: Post-Show Review

    Live From GSX: Post-Show Review

    This year’s Live From GSX program was a rousing success! Again, we’d like to thank our partners, and IPVideo, for working with us and letting us broadcast their solutions to the industry. You can follow our Live From GSX 2023 page to keep up with post-show developments and announcements. And if you’re interested in working with us in 2024, please don’t hesitate to ask about our Live From programs for ISC West in March or next year’s GSX. Read Now

    • Industry Events
    • GSX

Featured Cybersecurity

New Products

  • A8V MIND

    A8V MIND

    Hexagon’s Geosystems presents a portable version of its Accur8vision detection system. A rugged all-in-one solution, the A8V MIND (Mobile Intrusion Detection) is designed to provide flexible protection of critical outdoor infrastructure and objects. Hexagon’s Accur8vision is a volumetric detection system that employs LiDAR technology to safeguard entire areas. Whenever it detects movement in a specified zone, it automatically differentiates a threat from a nonthreat, and immediately notifies security staff if necessary. Person detection is carried out within a radius of 80 meters from this device. Connected remotely via a portable computer device, it enables remote surveillance and does not depend on security staff patrolling the area. 3

  • HD2055 Modular Barricade

    Delta Scientific’s electric HD2055 modular shallow foundation barricade is tested to ASTM M50/P1 with negative penetration from the vehicle upon impact. With a shallow foundation of only 24 inches, the HD2055 can be installed without worrying about buried power lines and other below grade obstructions. The modular make-up of the barrier also allows you to cover wider roadways by adding additional modules to the system. The HD2055 boasts an Emergency Fast Operation of 1.5 seconds giving the guard ample time to deploy under a high threat situation. 3

  • Luma x20

    Luma x20

    Snap One has announced its popular Luma x20 family of surveillance products now offers even greater security and privacy for home and business owners across the globe by giving them full control over integrators’ system access to view live and recorded video. According to Snap One Product Manager Derek Webb, the new “customer handoff” feature provides enhanced user control after initial installation, allowing the owners to have total privacy while also making it easy to reinstate integrator access when maintenance or assistance is required. This new feature is now available to all Luma x20 users globally. “The Luma x20 family of surveillance solutions provides excellent image and audio capture, and with the new customer handoff feature, it now offers absolute privacy for camera feeds and recordings,” Webb said. “With notifications and integrator access controlled through the powerful OvrC remote system management platform, it’s easy for integrators to give their clients full control of their footage and then to get temporary access from the client for any troubleshooting needs.” 3