Enhancing Security with Big Data Analytics

Anthony J. Ferrante


October 2, 2017

big data analytics SIEM

Security information and event management systems have long been the foundation for many organizations’ information security programs. While they remain an essential part of ensuring and maintaining strong cybersecurity, they need improvement to meet the demands of today’s rapidly evolving threat landscape.

Security information and event management (SIEM) systems actively and continuously collect and document security-related events across an organization, providing a security team with alerts about potential threats or vulnerabilities. Such systems are designed to provide a single view across the massive datasets generated daily regarding an organization’s enterprise security, based on parameters defined by an administrator to distinguish between normal and anomalous events. Security teams use this information to detect trends and patterns that may signal a threat. Today, however, an average large organization can generate hundreds of thousands or even millions of security alerts a day, making it virtually impossible for the information security team to determine which alerts point to real risks and which ones are meaningless. With no reliable or automated way to mine those alerts for actionable security intelligence, security teams are drowning in a sea of data with very little chance of finding the real threats.

Understanding the magnitude of damage cyberattacks can wreak—and the pace at which malicious actors are proliferating new malware and ransomware— has become important for every organization. Any savvy security professional knows that attackers only need to get it right once to do damage. The worldwide impacts of the WannaCry and NotPetya ransomware and the Mirai botnet are just a few recent demonstrations of the extent of the risk. New complications stemming from cloud use and big data are putting even more strain on both security teams and their SIEM systems.

Big data analytics offer one way to improve these security systems. When properly integrated with the SIEM system, analytics help make important determinations about incoming threats and internal vulnerabilities. With the ability to automatically sift through millions of security alerts to focus on the real threats, organizations can take a more proactive approach to defend against cyberrisks.
Improving SIEM

As companies adopt more cloud technologies and new apps are rolled out daily, SIEM systems are impacted twofold. First, in many cases, new data sources are not integrated into the SIEM software, so security events taking place in different applications may not be filtered through the system for notification. As a result, the SIEM essentially will not even know about these events.

Second, even if the data properly feeds into the SIEM, the software is often unable to handle that information or do anything with it. Outdated SIEM deployments, modules or patches will often simply ignore potential security events taking place in applications that are not properly integrated, and organizations can suffer gravely if those events include real attacks on the network. As in any big data effort, without the proper analytics models in place, the software cannot analyze the data and correlate it to any real, actionable information. The key to achieving value from SIEM alerts is to establish analytics modules that can feed in data, and to analyze those alerts in a way that correlates them to patterns of events.

Many SIEM systems are beginning to integrate predictive analytics modules, methods and capabilities to address these shortcomings. An increasing number of products on the market integrate big data analytics with SIEM systems. IBM, LogRhythm and RSA are a few of the companies developing data science applications for cybersecurity. Some developers are also using Hadoop and other open frameworks to integrate the results from analytics tools into the SIEM itself.

The level of adoption of these tools varies across industries. Organizations in industries that face the greatest cybersecurity risks, such as financial services, tend to have tighter integration than is typical in healthcare, manufacturing or other industries. And while SIEM analytics is currently the fastest growing category in cybersecurity, other pressing cybersecurity demands, tight budgets and lack of sophistication have all been barriers to widespread adoption.

It is important for companies considering deploying big data analytics for security purposes to understand that these tools can be costly, but without the right tools and models in place, there is no way to realize the benefits of the existing datasets. To get SIEM analytics up and running as an effective part of the security program, organizations can take some basic steps, including:

  1. Assess:  Conducting an in-depth assessment of the environment, including vulnerable and high-risk data and assets, is a necessary first step. With a map of the data and clear understanding of the critical network segments, the security team can then evaluate how various tools will meet their unique needs.

  2. Test: The assessment phase paves the way for further determination of the organization’s key weaknesses and, by extension, which data must be addressed. These areas can be set up as test beds for the analytics platform to begin to figure out which types of models need to be written to enable the SIEM system to absorb and understand the data to generate actionable intelligence.

  3. Gradually Expand: After the most critical segments are tested and the analytics are running smoothly across them, the platform can be broadened over time to include additional areas. This provides a reasonable, achievable way to eventually use analytics across the entire organization.

Human Intelligence

Once analytics tools are in place and effectively aggregating the SIEM alerts to identify patterns in security events, there are a handful of things security teams need to do to become more proactive about addressing threats. If, for example, the analytics call out the 30 most serious threats in a day and the details about them, the organization can shift from simply reacting to events to predicting which data remediation efforts or security controls should be put in place to develop a proactive defense.

Many fail to recognize that cybersecurity is actually a human problem. Cybersecurity threats are introduced by humans with a specific motivation, and as such, defenses must be powered by humans equipped with expertise and shared intelligence. Technology solutions can be applied on top of that foundation to enable better, faster and automated ways to execute security strategies.

Some organizations, however, will invest the time, money and resources to put an identification and analysis system in place, but then do nothing with the information gleaned. To ensure action is taken on such insights, organizations must have a well-defined intelligence program and process that combines information from internal analysis with external resources. Because everything in cybersecurity should be intelligence-led, this is where the human expert is particularly critical. Analysts who can understand the cyberthreat landscape, strategize actions based on new intelligence, and drive prevention, incident response and attack recovery are vital for a security program to succeed.

It is equally important to develop a platform that serves as the intelligence repository for the organization and enables sharing across the broader cybersecurity community. At the core of intelligence is information-sharing among many parties, which helps security teams understand a threat before it becomes a real problem. Many sophisticated global organizations have advanced information-sharing processes that include in-house platforms as well as third-party security providers to integrate information across various departments, organizations and industries. A security event a bank identifies today and a threat a government organization finds tomorrow can both provide valuable information that will benefit the broader cybersecurity community.

An integrated intelligence platform that can track and ultimately help mitigate cyberrisk may be purpose-built or customized by an organization. There are a variety of vendors that provide out-of-the-box software for this, such as IBM’s QRadar and Hewlett Packard ArcSight. MISP Threat Sharing is a community-driven, open-source threat intelligence platform that further encourages a connected and collaborative intelligence community. Most organizations will see the greatest value from a combined approach that includes third-party services that help establish processes, open-source standards and a customized in-house solution.
The Value of Information

Security teams must be prepared to fine-tune and manually modify their analytics tools as new capabilities are rolled out. SIEM systems can often generate false positives. When this happens, analysts will need to look at what is working in the model and what is not to reduce the false positives. Analysts or threat assessors can then dedicate more resources to looking at the actual events to validate them within the context of other threats and information in the intelligence-sharing repository.

Ultimately, these efforts help ensure that containment is in place around only the real threats, and that incident response and recovery can be easily and efficiently initiated as needed. Lessons learned from an event and its subsequent response activities can be fed back into the intelligence repository to refine the program to prevent the same events from happening again and to better defend against similar incoming threats.

Many organizations have prevented attacks from happening or reduced their impact by using a proactive intelligence-gathering approach. In the WannaCry ransomware incident, for example, intelligence analysts proactively collecting and sharing information slowed the damage. A 22-year-old analyst was researching command and control servers associated with the WannaCry malware and recognized that the next command and control domain in line would be used to continue spreading the ransomware. The analyst’s firm purchased the domain, ultimately thwarting the further spread of WannaCry. The NotPetya outbreak relied on similar techniques as WannaCry to propagate. While many firms were impacted, others that used lessons learned from WannaCry were able to avoid falling victim to NotPetya as well. This is a common way for the security community to incorporate big data analytics to find patterns of activity that they can use to defend against malicious actors.

Data overload is an issue for many corporations. Along with the challenge of new data sources, applications and IoT devices, the cyberthreat landscape is constantly evolving. Increasingly advanced threat actors are persistently engaging in campaigns to steal information or money from organizations or cripple their networks as a show of force. A proactive and predictive approach built on human intelligence and powered by advanced analytics is quickly becoming a reliable way for security professionals to get in front of hackers and implement effective security protocols.
Anthony J. Ferrante is a senior managing director and head of cybersecurity at FTI Consulting, and former director for Cyber Incident Response at the U.S. National Security Council and supervisory special agent at the Federal Bureau of Investigation.