The New Laws of Anti-Malware Technology

In a series of forthcoming blog posts, I am going to describe what I've termed "The New Laws of Anti-Malware Technology". The idea is that the face of malware has changed considerably, especially in the last five to ten years. But, unfortunately, far too many security vendors are still using an outdated approach that is several decades old. With each post in this series, I'll talk about one new "law" that I truly believe dictates how anti-malware technology needs to evolve to address the threats our customers actually face today and what we think they will encounter in the future.

New Laws of Anti-Malware Technology: #1 Malware Defense Today is Fundamentally a Big Data Problem

Since anti-malware technologies were first introduced decades ago, vendors assiduously performed some degree of sample collection, sample processing, detection generation, and detection publishing. In short, vendors had been tasked with the problem of translating back-office intelligence into customer facing protection. What has changed, however, is the sheer volume of data vendors must now deal with. The hundreds of threats companies dealt with on a daily basis less than a decade ago pales in comparison to the hundreds of thousands of threats they must handle on a daily basis today. Even worse, threats today are highly ephemeral. In fact, approximately 75% of threats we see today have a lifetime of zero (meaning that the first time we see them on a customer's endpoint is also the last time we see them). And in general, threat lifetimes can be measured in hours. These issues only appear to be getting worse.

Rather than fundamentally rethinking their approach, far too many vendors have simply tried to stretch the limits of their existing back-office infrastructures. While these vendors may have started off with a handful of back-office malware analytics machines, what they likely now have can best be termed as a messy patchwork of systems. Unfortunately, these systems are often disparate to the point where the underlying protocols, programming languages, databases, schemas, data formats, operating systems, operating procedures, and operational personnel are different. To make matters worse, personnel who might have initially developed some of these systems may be long gone. Even still, these systems were not all developed with the purpose of creating a tight feedback loop between in-field data and actual protection mechanisms. A data-driven approach must be a fundamental design principle from the onset — or the company will need to overhaul technical systems, deal with large-scale integration issues, and likely need to reorganize its operations significantly. Any attempt to become more data savvy post-hoc is riddled with issues, and is likely to be a non-starter in most cases.

We have been afforded the opportunity to take a clean slate approach, so we are not encumbered by the tremendous weight of legacy considerations. We replaced the dozens of systems other vendors use with literally just a handful. Furthermore, we established data models, schemas, and protocols that enable seamless correlation between these systems. The result is that we can harness the collective capabilities of these systems towards the problem of protecting our customers from the latest threats. Furthermore, this design has streamlined our process for translating back-office intelligence into customer facing protection. With our architecture, this process happens in near real-time – thereby providing immediate protection for our users. In contrast, for other vendors this process takes days (and sometimes even months!).

A data-driven approach also enables us to gather real-world metrics on how our technologies perform in the field. This type of view into actual field operations is unparalleled among existing offerings. For example, we are able to measure our true in-field efficacy, and also understand the respective contributions of our different detection technologies. Beyond giving us useful metrics, we also have real world data against which to measure new technologies that may be in the research or design phase. This, in turn, allows us to streamline our research, development, and quality assurance processes because we do not have to “shoot in the dark” as many other vendors are unfortunately forced to do.

Thinking of malware defense as a big data problem is a fundamental paradigm shift, and taking this approach is an absolute requirement given the nature of today's threats. While traditional methods had their time and place decades ago, a changing of the guard is necessary for providing the level of protection and control that customers now need.