Threat Hunting Program: 5 Best Practices for Success
There was a pretty significant statistic that was recently released in Mandiant’s M-Trends 2022 report. In it, they cite that the median number of days an attacker resides in a system before detection (the “dwell time”) fell from 24 days in 2020, to 21 days in 2021. On the surface, that statistic may seem encouraging, especially with the pervasiveness of tools that help visibility like EDR, NDR, and XDR. But as it also says that adversaries — lead by ransomware actors — are becoming less concerned with stealth, and more goal-oriented. It is also telling that even with these advanced tools in place, adversaries are still able to evade detection, and security teams are often left to sift through the wreckage and find out how bad the damage really was.
This is especially troubling for MSSP and MDR providers — who are often entrusted with monitoring dozens or hundreds of clients. Are you confident that your security teams are able to detect the adversary’s initial point of access when 2021 saw a record number of zero days being used in the wild? Relying on reactive security technologies to alert you to a breach often leaves security teams a day-late-and-dollar-short. This is where a threat hunting program can enable MSSPs and MDR providers to more proactively search through their clients’ environments to identify threats that have evaded security controls but before they can achieve their nefarious objective.
WHAT IS A THREAT HUNTING PROGRAM?
Suffice to say, threat hunting is a term that gets used a lot in cyber security. Like… a lot. And you’ll often hear words like “proactive” and “iterative” get thrown around alongside it. Unfortunately, many of these definitions are a bit too light on actual details.
Threat Hunting is a security methodology that is either hypothesis- or data-driven and searches for unknown threats. We’ll get to what these mean in a second, but the important takeaway is what threat hunting isn’t — and that is IOC-based. Threat hunts will not start with, but they will generate, IOCs. This is because IOCs represent known threats, and if they are known, you shouldn’t be hunting for them, you should be blocking them.
Data-driven hunting starts with hunters pouring over log files often looking outliers or anomalies. This type of hunting is often where organizations first get their feet wet with threat hunting, but it can often be overwhelming as it can feel like hunting for a needle in a haystack. Hypothesis-driven hunting is more targeted. It begins with a broad question, and through a series of refinements, it is narrowed down to a specific hypothesis usually centered around a specific behavior (often referred to as a TTP). A hunt team will then comb through an environment looking for that specific behavior to see if an organization has been affected.
Now another key element to the definition of threat hunting is that it is not tool-based: threat hunting is people and process-based. Just because an organization has a threat hunting platform doesn’t mean they are hunting. But if you have the people and processes in place, you can hunt using almost any security tools. Despite this, most organizations struggle in the early stages of setting up a threat hunting program. To help counter this, we’ve put together 5 best practices every MSSP and MDR provider should keep top of mind when setting up a threat hunting program.
5 BEST PRACTICES FOR A THREAT HUNTING PROGRAM
IN A THREAT HUNTING PROGRAM, VISIBILITY IS KEY.
As we have said, a threat hunting program doesn’t rely on a specific platform or tool, instead it relies on people and processes. However, you can significantly improve the results of your threat hunting program by ensuring your hunt teams have access to the right log data. If your organization has mostly focused on traditional block-and-tackle security, this means you are probably going to need to start ingesting a lot more log data. Afterall, threat hunters can only hunt what they can see. This is going to require log data from your security tools, endpoints, and network data. If you want to deep dive into this, we’ve built a pretty comprehensive list for the best log data at the endpoint- and network-level.
BASELINING IS HARD, BUT IT IS WORTH IT FOR THREAT HUNTING PROGRAMS!
Another very important element to a successful threat hunting program is a process called baselining. This is something that every security team should be doing, to some degree or another, but for threat hunting it is absolutely crucial. This is because the behaviors hunters are on the trail for are not always malicious, sometimes they are just suspicious.
Is that Excel spreadsheet accessing an AWS endpoint the first stage of a malware attack?
Or is it a power user that has automated a time-consuming part of their job?
Threat hunters use baselining to determine what is normal in an environment. While this can sound like an arduous task, especially for MDR or MSSPs that might have dozens of clients or more, the good thing to note is that this will be an ongoing practice, so it isn’t a pre-requisite to threat hunting, but more of a co-requisite. We do recommend using a collaborative platform to document these on-going findings to help with knowledge transfer.
NEVER STOP HYPOTHESIZING
As security teams mature their threat hunting program, they often start incorporating hypothesis-driven hunting into the mix. As we mentioned, this type of hunting will start with the assumption that a network has been compromised, using a particular vector, and will eventually narrow down to what “behaviors” you would expect to see from that adversary.
This is where a lot of organizations stumble initially and end up falling back on IOCs.
This often happens because teams aren’t sure where to derive these behaviors from, especially because a lot of threat intelligence “reports” don’t mention them. This is where hunters need to draw from a wide array of experience, including previous experience and past incidents. For MSSP and MDR providers this can be especially challenging if their clientele spans across industries, sectors, and geographies. This challenge can be multiplied when threat hunt teams are faced with the task of not just developing, testing, and deploying hunts, but keeping them-up to date as well. In this case, or for teams that are just starting out, threat hunting content platforms can help by providing a broad spectrum of threat hunting content that is developed and tested and can be deployed instantly across security tools from various vendors. For instance, a Community account on Cyborg Security’s HUNTER platform, for instance gives dozens of threat hunting packages for some of the most common adversary behaviours, totally free! This can give hunt teams the breathing room they need to get up-to and keep up-to-speed with the constantly evolving threat landscape.
USING HUNTING AS A FORCE MULTIPLIER
A common mistake that security teams make when they first start out threat hunting is to look at threat hunting as an island unto itself. The team hunts, finds, and remediates. The reality is that threat hunting should be used as a force multiplier. When hunters identify something that slipped through, it is critical that actionable detection content be created so that in the future the SOC teams can respond appropriately, and hunters aren’t spending their time hunting the same threats over-and-over.
DON’T JUST “CHASE THE SQUIRREL”
A challenge that a lot of security teams can face is the temptation to only “chase the squirrel” when threat hunting. This happens when a new high-profile vulnerability, malware, or exploit is released, often making the mainstream news media, and especially for MSSPs and MDR providers, is usually accompanied by a flurry of emails and phone calls to find out if clients have been impacted. While this is a tale, in security, as old as time, the reality is that it is also the least efficient use of threat hunting resources. This is because hunt teams should be actively hunting for the behaviors that an adversary would exhibit once they get into an environment. Can they hunt for exploitation of a particular vulnerability?
Absolutely.
But especially in the “fog of war” that often surrounds these big events, often not all the details of the attack are fully known and fleshed out. But if you layer active monitoring for the exploitation (usually done by a SOC team) with hunting for the known behaviors the adversaries exhibit once they get past the perimeter makes your organization’s security all the more robust.
CONCLUSION
MSSP and MDR providers have become a crucial link in the chain for organizations’ security. And with this has come the realization that traditional reactive security just isn’t enough anymore — and that there is critical need to deliver advanced services like, threat hunting, in order to keep adversaries at bay.
