Security at Scale—With Great Success Often Comes Great Security Problems

Some common areas and issues to keep growing your business, not your security risk.

• By Gary Brown
• Apr 26, 2019

All businesses want to become the next big thing, but with great success often comes greater security issues. How does a company scale for growth without becoming a security risk? What are the best things to do to maximize efficiencies, and to properly scale your security practice? 

Here are some common areas and issues to keep growing your business, not your security risk.

Plan for Growth

Always design your security architecture so that it can scale as needed. Ensure that network maps and asset inventories are correctly maintained and are easy to access by the security team. There’s nothing more frustrating than to be in the middle of an incident and your team is struggling to identify what data is on which machine and who is responsible for it.

Operations

Anyone who has run any sort of security group of any size knows that having a well planned out operations groundwork is critical to any sort of effectiveness. Formal run books, incident response plans, formal procedures for commonly encountered issues, and so on. When a machine gets popped with ransomware, who is on the hook to remediate, and who does the investigation to find the root cause? As your company grows, so will the security team’s reliance on other teams like networking and IT. Ensure that you integrate and train them on your processes, so that not only do incident responses run more smoothly, but the teams learn to work together. Having a great relationship with the actual boots on the ground can be priceless, especially when the you know what hits the fan.

Automation

Once you start to scale your network, you can be pretty much assured that your security team and resources won’t scale as quickly. This usually means that you need to lean on automating as many tasks as possible. For example, if you require new passwords frequently, with a million plus users, you had better have an automated way to handle password resets. Self-serve processes are great, but as always, ensure that any scripts or services you deploy are secured – especially if they are handling sensitive tasks.

Wherever possible automate patching, automate scanning, and implement an effective remediation assignment and tracking. Let’s be honest. Any IT organization that isn't good at automation will fail badly at scale, no matter what it is, security or not.

Proper Monitoring

Proper security monitoring can encompass a lot of things Host IDS, FIM, end-point security, network IDS, threat intelligence and so on.

• Monitoring the appropriate traffic with the proper people is critical. Don’t deploy kids fresh out of school in the Security Operations Center (SOC). Make sure there is at least one pair of experienced eyes looking at alerts. Too often SOC analysts, through lack of experience, will flag individual incidents as minimal, while not recognizing they are part of a concerted attack. 

• Ensure that you have logs from everything, and that outages are remediated quickly. Don’t just focus on external traffic – focus as much energy on detecting anomalous internal traffic for signs of employees poking around where they shouldn’t be, as well as possible breaches.

• As you scale, consider using honeypots. These can attract attackers, and when properly configured can yield extremely valuable information.

• Wherever possible, ensure that local logs are monitored; too many organizations focus on general network traffic, ignoring local logs. The result is often compromised machine going undetected for a long while

• Ensure that all important sources of threat intelligence are being centralized so that your SOC analysts can get a big picture view.

Reduce the Noise

Reducing the noise is perhaps the most critical change one can make to their network. Too often the approach of security teams to false positives is to tune them out at the SIEM and move on. There’s a big problem with this approach. First, there’s unnecessary traffic traversing the network and in the logs, but more important, the underlying issue isn’t being addressed. The best approach is to remediate, limit, then filter. 

Remediate is the preferred approach; actually fix what’s making noise. This plays into the operations part above; as you grow, make sure your incident response plans include IT and networking. When you see a machine innocuously repeatedly trying to reach a service that it is blocked to, have IT investigate and fix it. This is also an opportunity for the security team to illustrate to the IT and networking groups that security can be an effective adjunct to their teams, identifying broken systems well before their groups will.

If the false positives can’t be remediated, wherever possible, block the alerts at the source. This limiting of unnecessary traffic will ease your analyst load, and will help reduce the traffic being thrown into your security logs and appliances. Finally, if you are unable to remediate or limit the traffic, filter it at the SIEM, but not until you’ve exhausted the first two option.

Reducing the noise isn’t just limited to reducing unneeded network traffic; it’s important that things like threat intelligence feeds are also fine tuned to your environment. If you don’t run Apache, your analysts don’t need to be seeing any of those intelligence alerts in their inbox. Your end goal should be to minimize the hay, making the needles much more apparent.

No matter your organization’s size, if you follow these basic concepts, your security processes should run more smoothly, and growth will not be an issue. A little pain when you are small is much better than a lot more pain down the road.