Surprised By Storage

Megapixel cameras bring higher resolution images, but the huge amount of data they generate presents new challenges for IT and security pros

• By Sharon J. Watson
• Aug 17, 2007

"I am not a tech guy," said Ken Haverlan, director for security, health and safety at Middletown County Schools in New York. That's why he immediately brought Mike Tuttle, the school district's IT chief, into discussions about a new IP video surveillance network.

Although Haverlan's primary goals were capturing identifiable faces and ensuring video was immediately available to first responders, Tuttle recalls his initial question: "How am I going to store all this footage and for how long?"

Capturing a definitive identity on video and storing that video may seem like two distinct issues. Yet, video storage is a critical issue shaping how enterprises make the transition to IP-based surveillance networks.

As security departments replace VCRs and videotapes with digital recording devices, digital cameras and encoders to transform analog camera output to IP, they and their IT counterparts quickly learn that networked video generates a lot of data -- potentially terabytes of it every day.

That's data that must be managed, and IT departments are assuming the responsibility.

"When the data's on our network, we know how to get it, store it and back it up," Tuttle said. "Ken relies on us for connectivity, availability and troubleshooting."

Security departments in some industries, such as casino gaming, can still make a case for running their own networks and storage.

"More often, as video networks evolve from closed circuit to IP, they become part of the existing IT infrastructure, although they may be partitioned as a subnetwork or a separate storage area network within the security department's purview," said Dick O'Leary, senior director for EMC Corp.'s physical security solutions in Hopkinton, Mass.

Such convergence can be driven by regulatory requirements or corporate data storage policies, economies of scale and plans to make video data available to other enterprise applications.

Further, choosing switches and routers and designing network storage topologies requires networking expertise most security officers don't have. Similarly, most IT professionals are not qualified to decide placement and angles for cameras or create access and use rules for them.

"You definitely need to work simultaneously as one organization," Tuttle said of balancing IT requirements with security's needs.

Less Bandwidth, More Storage

That's especially true because decisions about how much video to store, for how long and where to store it can affect where, which and how cameras are deployed.

In general, IT and security professionals underestimate storage requirements, vendors say.

"Video will require less bandwidth and more storage than they expect," said Fredrik Nilsson, general manager for Axis Communications-North America, headquartered in Lund, Sweden, with U.S. offices in Chelmsford, Mass. He and other vendors estimate that storage costs run as much as 30 percent of a video project.

Though storage devices are steadily becoming cheaper, video is data-intensive, and the higher a camera's resolution, the more data it generates. For example, Bosch Security calculates that a single camera transmitting at 30 fps using the common interface format resolution of  704 x 576, or 4 CIF, generates 11 gigabytes of data per day. Three hundred of those cameras recording continuously would spew out 3,300 GB each day, or more than 3 Terabytes of data.

Most users on a budget soon drop expectations of running megapixel cameras constantly at 30 fps.

"They would like to have higher resolution if they could afford it," Nilsson said.

Smarter Video Collection

Users can reduce short and long-term storage needs while still capturing critical video data in several ways. First, some cameras and controllers are intelligent enough to run at 1 to 4 fps until they sense motion or another event trigger, then ramp up in an eyeblink to a higher frame rate. These cameras can be deployed in less sensitive locations, while higher resolution digital cameras running at higher frame rates are installed at critical points.

Video compression technologies are improving too, reducing the amount of data to store while maintaining image integrity.

"H.264 compression reduces file storage requirements significantly," said Mike Morper, director of product management for GE Security's Digital Surveillance Video in Costa Mesa, Calif., noting that the industry compression standard improves storage efficiency by 40 to 60 percent. "It does things that are smart for video streaming."

Content analysis, done in smart cameras, video management systems or separate analytics programs also can reduce data volumes by determining what video data must be stored at what resolution.

"If nothing's moving, you can dump 90 to 99 percent of the data," said Stephen Russo, director for security and privacy technology at IBM Global Technology Services, Armonk, N.Y.

"Move as much intelligence to the edge of the network as possible, and you'll have fewer bandwidth and storage issues," said Leon Chlimper, vice president of systems for Bosch Security Systems, based in Fairport, N.Y. As cameras and analytics on the network's rim get smarter, he expects users to save bandwidth and storage by transmitting only video that needs to be seen.

Where to Put it All?

Many camera, software and storage vendors support the idea of moving video storage to the network's edge. Security and IT chiefs usually agree, say vendors.

"You'll always have a need for storage on the network edge," Morper said. Physical security departments like having data stored locally in case the network should fail.

Local storage also offers users the flexibility to deploy some higher resolution cameras without necessarily tying up bandwidth. For instance, Avigilon, a Vancouver, British Columbia-based maker of megapixel cameras, runs its high-resolution video output to a high-capacity redundant array of inexpensive disks at the network edge. Clients using the cameras to augment analog cameras may run an analog signal out of RAID storage to DVRs or a video management system.

In its edge storage solution, Steelbox Networks Inc. uses a 3-foot-long cable to attach cameras from a variety of manufacturers directly to its Digital Matrix Storage Switch controller and storage appliances of up to 16 Terabytes.

"We store it all and firewall it, so you only get video on the network that you want to see," said Chip Howes, president and CEO for Atlanta-based Steelbox.

Bosch Security, Cisco, GE Security and IBM, among others, also offer storage devices that can be deployed on the network's rim. Other vendors, like On-Net Surveillance Systems Inc. and IndigoVision, run their software programs on off-the-shelf workstations to manage primary video.

"There's no point buying an expensive disk array for streaming video," said Gadi Piran, president and CTO of OnSSI, based in Suffern, N.Y. OnSSI uses a workstation's storage capabilities to hold about an hour's worth of incoming video for analysis; the filtered video is offloaded to archival storage. This buffering balances the network load, provided the user has sufficient server capacity for the real-time video.

As for long-term storage, analytics keep video volumes manageable.

"We're being asked increasingly to prune and archive data," said Steve Collen, director of product management for Cisco's physical security business unit, based in San Jose, Calif. He and other vendors say users want integrated analytics so they can make informed decisions about what video to keep.

For example, IBM analyzes video as it compresses it to create metadata. Only this high-level, descriptive information, along with a bit of the beginning and ending of the indexed video, is stored in a central relational database, Russo said. The complete video described by the metadata is indexed and offloaded to a separate storage device that can essentially reside on any network.

Not Just for Security Anymore

All vendors emphasize their open hardware platforms and application programming interfaces to ensure storage devices work with whatever analytics and other security or business programs an enterprise is running.

Open systems are important because many enterprises plan to integrate video data not only with security applications like access control, but also with traditionally business-oriented applications such as point-of-sale analysis and training.

"The industry is just starting to see video as data," said GE's Morper. He and other vendors say it is possible business users might want to use the surveillance network to study how customers behave in front of specific sales displays, to name just one application.

"As more digital video is deployed, people will come up with interesting ways to use it," said Jame Ervin, product manager for Stonefly Networks in San Diego, Calif.

Security chiefs might initially resist commingling security and business video, but could find that's one way to grow a bigger budget and more support for their efforts.

CSOs can show operating and marketing officers how the video infrastructure can net valuable marketing and training data, said Jeanne Jang, IBM global leader in digital video surveillance. "It's beyond traditional security, so other important players on the management team can support it."

Thinking about the potential business impact of video system and storage does add another layer of critical planning complexity.

"In this day and age, sitting down with only IT and security at the table is a mistake," Chlimper said. "The systems can do so much more."