No Protection From Bad Data

No Protection From Bad Data

Beware what the experts claim as noise and a security signal

Walking the expo floor at the most recent RSA conference, it was hard to miss how many companies were talking about big data. Maybe they used the word analytics, or perhaps they called it machine learning. The claims were all similar. Give me your tired, your poor, your huddled masses of log and incident data that you’re yearning to make actionable—or something along those lines. They all wanted to sell us on the notion that big data can take operational noise and turn it into security signal.

Of course, the next line of the famous poem I misquote refers to “wretched refuse,” and that’s a pretty good description for most of the data people would feed into these systems. The belief that such systems can take refuse and spin it into gold is magical thinking. Big data, data science, machine learning and other novel approaches do have true promise for security leaders and practitioners looking for better results. There are ways to get better data to feed these new systems if you know where to look. When you combine good data and big data, you get results that can seem like magic without having to wear the funny hats.

The need for good data starts right in the beginning of the big data lifecycle. The real power of these analytics systems are in the models they build. A good model will allow you to go from a reactive to a predictive approach to security incidents. It’s like having hindsight for the future—you’re always seeing the signs 20/20. But how can you tell if you’ve got a good model? The model needs to be tried out on a dataset you know a lot about.

This is the practice test where you have the answers and you want to see how well you may do on the real test later. Of course, a lot of the vendors have done the dirty work for you here. They have models that they feel should work well.

Best practice would be to take their model for a spin on your well known data and see what happens. Maybe the model needs some adjustments for the particular configuration of your IT infrastructure. People paying attention often get stuck right at this phase. Do you have a good backlog of data you could use to run tests like this? Are you archiving the critical logs and other data sources that these models would employ? Have you done the forensics to identify where the data you have corresponded to breaches or incidents you’ve experienced so you know what the models ought to turn up? Too often the answers to these questions are no, no and no.

Let’s say you get to the stage where you have models that will work. The next challenge on the path to big data nirvana will be having these analytics cough up the answers to your burning questions. Here’s where another problem in the data often emerges. You’ve set up your shiny new data science driven machine, but suddenly you find the questions you’re being asked and the data you have don’t line up.

It’s often a

level of abstraction mismatch. The executives ask questions about people, but your data is about machines and IP addresses. This can also affect your modeling. If you only have a hammer, you smash everything like a nail. Maybe you go find a bunch of data about people, but if you don’t adjust the models so that they treat people, machines, IP addresses and everything else like they ought to be treated and as if they are in the right relationships, then you’ve made the problem worse and not better. Often bad models with data at different levels of abstraction (e.g. a person and an IP) that fall into some correlation will start to crank out tons of bad conclusions when one thing does something normal for its level of abstraction (e.g. a person quits) that doesn’t have anything to do with the thing it fell into accidental correlation with.

An excellent example of good modeling for these disparate types of data in the security space is how Securonix has used peer grouping from Identity and Access Management (IAM) systems to enrich their data. Like many in the User Behavior Analytics (UBA) space, Securonix will look at lots of different data sources to get you the insight you want about what your users are up to. They could have been prone to the troubles of modeling with different layers of abstraction and correlating at the wrong levels, but seem to have gotten it right.

When they take in data about people and their organizational relationships from IAM systems, they use this to influence the outcomes not by directly correlating it with activity at the network and system layer, but rather by augmenting expectations of what is normal based on the peer group you may have. A peer group will be people in the same building, managers at the same level of the hierarchy, or employees with the same job role.

You would expect these people to have notably similar activity patterns, and you can use that to learn what they do and point out anomalies. That’s getting the analytics game right by pulling in the right data and using it well.

Sometimes you have to work with what you’ve got, and that means bad data will happen to you. Nothing serves as a better example for that than the notoriously poor quality logs on Microsoft platforms. Single events like a bad authentication appear as multiple entries in multiple events on multiple systems. Often, even if it’s possible to collect all the separate events, you find key pieces of data like an IP address is missing so correlating that event with other events from other sources is near impossible.

The default level of logging doesn’t offer enough detail for proper detection or forensics from a security perspective. Turning up the logging so that it becomes nominally useful for security means sacrificing a huge chunk of the system resources. To add insult to injury, collecting and parsing all these logs is immensely burdensome. If you’re a practitioner, then none of this is news to you. From the analytics perspective, these problems often hide behind the SIEM system. You don’t know you’re dealing with such low quality data until you start using it via the SIEM and getting poor results.

This is where you will likely need to look to other sources for good data if you want the quest for big data analytics to have a happy ending. The STEALTHbits StealthINTERCEPT platform is able to get all the security data you want from these Microsoft platforms and overcome these native logging issues. You can plug it into the SIEM, feed directly from Stealth- INTERCEPT itself, or even plug it into something else that can consume our SYSLOG output. This means adding another layer to your infrastructure, but the result is a high quality, real time stream of security events from your Microsoft systems. I’m picking on Microsoft, but it’s not like they’re the only one guilty of having bad logging. Luckily for many of these problem children you’ll find other solutions like ours that will help fill the gaps and get you the good data you want.

Moving from reactive controls to predictive controls has been a goal on the horizon for security organizations for a long time. Solid models powered by sound data science that processes the huge well of big data can finally make security predictive. That’s assuming they’re fed by streams of good data. If the stream of data is polluted, then all that water in the big data well isn’t worth the trouble to drink. You’re going to need good data to get these systems going, which means discipline and practices around retention you may need to improve.

You will have to be sure you’re getting all the right data from the right level sources and putting them in just the right relationships in your models so they produce useful results even as the real world changes around them. Sometimes you’re going to need to make tough choices to invest in better data sources or deal with lower quality results. But if you can get the data right, then big data will do right by you.

This article originally appeared in the August 2015 issue of Security Today.

Featured

  • New Report Reveals Top Trends Transforming Access Controller Technology

    Mercury Security, a provider in access control hardware and open platform solutions, has published its Trends in Access Controllers Report, based on a survey of over 450 security professionals across North America and Europe. The findings highlight the controller’s vital role in a physical access control system (PACS), where the device not only enforces access policies but also connects with readers to verify user credentials—ranging from ID badges to biometrics and mobile identities. With 72% of respondents identifying the controller as a critical or important factor in PACS design, the report underscores how the choice of controller platform has become a strategic decision for today’s security leaders. Read Now

  • Overwhelming Majority of CISOs Anticipate Surge in Cyber Attacks Over the Next Three Years

    An overwhelming 98% of chief information security officers (CISOs) expect a surge in cyber attacks over the next three years as organizations face an increasingly complex and artificial intelligence (AI)-driven digital threat landscape. This is according to new research conducted among 300 CISOs, chief information officers (CIOs), and senior IT professionals by CSC1, the leading provider of enterprise-class domain and domain name system (DNS) security. Read Now

  • ASIS International Introduces New ANSI-Approved Investigations Standard

    • Guard Services
  • Cloud Security Alliance Brings AI-Assisted Auditing to Cloud Computing

    The Cloud Security Alliance (CSA), the world’s leading organization dedicated to defining standards, certifications, and best practices to help ensure a secure cloud computing environment, today introduced an innovative addition to its suite of Security, Trust, Assurance and Risk (STAR) Registry assessments with the launch of Valid-AI-ted, an AI-powered, automated validation system. The new tool provides an automated quality check of assurance information of STAR Level 1 self-assessments using state-of-the-art LLM technology. Read Now

  • Report: Nearly 1 in 5 Healthcare Leaders Say Cyberattacks Have Impacted Patient Care

    Omega Systems, a provider of managed IT and security services, today released new research that reveals the growing impact of cybersecurity challenges on leading healthcare organizations and patient safety. According to the 2025 Healthcare IT Landscape Report, 19% of healthcare leaders say a cyberattack has already disrupted patient care, and more than half (52%) believe a fatal cyber-related incident is inevitable within the next five years. Read Now

New Products

  • Camden CM-221 Series Switches

    Camden CM-221 Series Switches

    Camden Door Controls is pleased to announce that, in response to soaring customer demand, it has expanded its range of ValueWave™ no-touch switches to include a narrow (slimline) version with manual override. This override button is designed to provide additional assurance that the request to exit switch will open a door, even if the no-touch sensor fails to operate. This new slimline switch also features a heavy gauge stainless steel faceplate, a red/green illuminated light ring, and is IP65 rated, making it ideal for indoor or outdoor use as part of an automatic door or access control system. ValueWave™ no-touch switches are designed for easy installation and trouble-free service in high traffic applications. In addition to this narrow version, the CM-221 & CM-222 Series switches are available in a range of other models with single and double gang heavy-gauge stainless steel faceplates and include illuminated light rings.

  • PE80 Series

    PE80 Series by SARGENT / ED4000/PED5000 Series by Corbin Russwin

    ASSA ABLOY, a global leader in access solutions, has announced the launch of two next generation exit devices from long-standing leaders in the premium exit device market: the PE80 Series by SARGENT and the PED4000/PED5000 Series by Corbin Russwin. These new exit devices boast industry-first features that are specifically designed to provide enhanced safety, security and convenience, setting new standards for exit solutions. The SARGENT PE80 and Corbin Russwin PED4000/PED5000 Series exit devices are engineered to meet the ever-evolving needs of modern buildings. Featuring the high strength, security and durability that ASSA ABLOY is known for, the new exit devices deliver several innovative, industry-first features in addition to elegant design finishes for every opening.

  • Connect ONE’s powerful cloud-hosted management platform provides the means to tailor lockdowns and emergency mass notifications throughout a facility – while simultaneously alerting occupants to hazards or next steps, like evacuation.

    Connect ONE®

    Connect ONE’s powerful cloud-hosted management platform provides the means to tailor lockdowns and emergency mass notifications throughout a facility – while simultaneously alerting occupants to hazards or next steps, like evacuation.