Detection Content — The Trouble with Free

By Josh Campbell From Cyborg Security

As organizations continue to mature and grow their threat hunting capabilities, especially as they incorporate more structured threat hunting into their operational cadence, a topic of concern for many of them is detection content. Detection content, sometimes referred to as queries or use cases, is a topic that we have explored before. One of the questions that is often asked, however, is where do organizations get their detection content?

CONTENT SOURCES

Cyborg Security recently conducted a poll of nearly two hundred organizations to see where those organizations were deriving their detection content from. The results are below, and they may surprise you.

By a landslide, the majority of organizations (64%) primarily used open source repositories for threat hunting and detection content. Typically, this content is built by security researchers, and is often freely available on personal blogs, independent research, and code sharing repositories.

However, perhaps more interestingly is only 17% of organizations primarily rely on in-house content development. This likely indicates that for organizations with in-house content development personnel, they are mostly working on re-engineering and customizing content from those open source repositories. What is, perhaps, more worrying however, is that for organizations without in-house content development, a huge portion of their detection content appears to be coming from those free sources.

THE TROUBLE WITH FREE

Let’s be very clear: that security researchers donate their time, cycles, and knowledge to building detection content is an example of what makes the infosec community an amazing group of people. So, when we talk about the trouble with free detection content, we are not, in any way, detracting from their work, nor doubting their abilities.

Instead, the problem is that so many organizations seem to rely very heavily or, even exclusively, on that free content. Much like leveraging open-source code in commercial operations or products, there can be some stumbling blocks that organizations should keep in mind lest they bake in too much security debt into their security operations.

STALE CONTENT

One of the major problems with free detection content is that it can quickly become out-of-date or stale. Malware authors are continuously motivated to improve their existing “products,” both to add new features as well as to help their creations remain undetected. This means that malware will likely change and frequently. However, a lot of freely available detection content is maintained as a “best effort,” if at all. The result is that out-of-date content may detect older versions of a malware but leave organizations exposed to newer variants and give organizations a false sense of security.

CONTENT VALIDATION

Another, similar problem, that organizations which rely heavily on free content can face, is that there may be an assumption that thorough testing and validation was done to ensure the detection content performs — and detects — as one would expect. For example, if a researcher looks to detect a particular strain of malware, they may write detection content for it. However, they may be unaware that other versions of it even exist. This again can lead organizations to adopt a false sense of security that their content can detect everything they think it can.

THE RE-ENGINEERING EFFORT

Perhaps one of the most common problems with free content is simply re-engineering. Even if the content was engineered for an environment nearly identical to another organization, there is likely to be subtle, yet impactful, differences in the data. This means that organizations must spend considerable time and resources — if those resources even exist — to re-engineer the content to tailor and optimize it to the environment. This can mean delays of days, weeks, or even longer depending upon priorities, exposing organizations to additional risk.

DOCUMENTATION! WHAT DOCUMENTATION?

Lastly, another extremely common problem with relying on exclusively free content is that often it was written with a particular analyst skillset in mind, namely the author, and with little accompanying documentation. However, the reality is that analysts’ abilities and competencies will differ, and this means that analysts may have to “fill in the blanks,” and different analysts may fill in the same blank differently. This can result in a lot of extra work, as more skilled analysts have to create procedural documentation for this free content if they want their triage and analysis to be rigorous and repeatable.

CONCLUSION

Again, it is important to note how fortunate the infosec community is to have such talented minds as those that contribute to the security of the whole. However, it is concerning that so many organizations still rely so heavily on free detection content, especially when those same organizations would be leery of relying on open source projects in other production environments.