Rethinking Email Threat Detection Models for Enterprises [A Deep Dive]

Contrary to what millennial work and communication habits may suggest, email is not dead. It is still an important part of both personal and enterprise communications.

Email is not only used to communicate internally; it is also employed in marketing and correspondence with investors and business partners. Nothing compares to its familiarity and versatility as a mode of communication. Used in both formal and informal situations, it can contain lengthy text content and relatively large file attachments of different types.

It is this popularity and flexibility, however, which make email a common target for hack attacks. A large chunk of phishing activities and malicious software distribution are made possible through email.

Businesses acknowledge this reality, so they implement security measures compulsorily. In fact, many enterprise-grade cloud platforms automatically scan emails for malware.

Unfortunately, though, detecting and preventing threats is far from easy, especially when a security system encounters a new threat for the first time.

High Miss Rates During First Encounters

First encounters make email security tools suffer notably high miss rates. The leading security tools at present are generally effective, unless they face off with unknown attacks or those that they come across for the first time. These are among the conclusions that we can draw from BitDam’s newly published study on the effectiveness of email security systems.

The BitDam study found that email security solutions such as Microsoft’s Office365 ATP and G Suite Enterprise fail to identify or stop threats properly during first encounters, 20% to 40% of the time.

Worse, it takes 24 to 48 hours for security systems to become adequately acquainted with unknown threats and stop them effectively. That’s an extremely long time in terms of cyber security, since it only takes seconds or minutes for email attacks to penetrate successfully.

Imagine the amount of ransomware, spyware, adware, and deceptive schemes that make their way into individual computers or networks when threats are left undeterred for more than a day.

The high detection failure rates when dealing with unknown threats and the long time-to-detect (TTD) after the first encounter are what make it necessary to rethink email threat detection models for businesses. Security firms need to improve the strategies they use to protect emails in light of the increasingly growing number of new or otherwise unknown cyber threats.

Prevailing Models of Email Security

Most security systems protect emails through threat signatures, sandboxing, and machine learning, among other techniques. They collect signatures or identifiers which are used to determine if a file or activity is harmful or anomalous.

On the other hand, security tools employ sandboxing to isolate applications from critical systems. This prevents malicious software from readily accessing system resources whenever they manage to penetrate. Modern email security systems also integrate machine learning to enable more efficient detection and prevention of threats.

They can automatically learn how to make educated guesses as to whether something is a threat or not based on the historical data they compile.

These methods generally work well, but tests show they have a fatal defect: unknown threats. The rise of unknown or unidentified malware make email security less reliable. Even industry leading email protection systems have this Achilles Heel.

The reality of diminished effectiveness in filtering threats is demonstrated by the BitDam study mentioned earlier. This empirical study that commenced in October 2019 tested Microsoft’s Office 365 ATP and Google’s G Suite Enterprise.

Diminished Effectiveness and the BitDam Study

The study entailed the continuous gathering of new malicious files and modifying them to produce variants that serve as new threats. The newly collected malicious files, along with their variants, were then sent to emails protected by Office 365 ATP and G Suite. The email accounts were monitored, and information on missed detection and TTD were recorded and analyzed.

All of the malicious files sent to the protected emails were verified to be malicious to make sure that a failure of detection was indeed a failure and not just a case of mistaken identification. If the harmful files were not detected, they were sent again until they are detected and stopped.

The test conducted by BitDam found that Office365 ATP had miss rates ranging from 15% to 31% over a seven-week period, with an average miss rate of 23%. G Suite didn’t fare any better, as it recorded an average miss rate of 35.5%. Its worst performance was on the first week, recording a miss rate of 45%.

When it comes to TTD, Office365 had an average of 48 hours while G Suite clocked in at 26.4 hours. This means that it takes them an average of two days and one day, respectively, to correctly detect harmful files they initially missed, when these undetected malicious files are re-sent.

Unavoidable Gaps in Detection

It’s shocking to know that the most reputable names in the internet fail to detect threats by as much as 45%, and that it takes them more than a day to retrain their defenses and successfully intercept threats they erroneously classified as safe earlier. Less popular security solutions would likely perform considerably worse when subjected to the same tests.

Of course, security systems are not expected to immediately detect threats on the first encounter. There can be up to three gaps in detection until a threat is correctly spotted and stopped.

The first is from the first encounter to the time a reputation service (or threat hunting) is employed to match a potential threat to a database of signatures. This is the basic process in threat detection. Harmful attachments are identified based on data gathered from various sources.

The second gap is from the time the threat hunting stage fails to correctly identify the malicious file to the time another stage is undertaken to qualify the maliciousness of a potential threat. For example, a potentially anomalous attachment may be allowed to pass through because it is deemed to be merely a marketing campaign. A deeper examination may find serious anomalies disguised by the marketing front.

The third gap appears when the threat remains undetected after the second gap until a new mechanism is added to detect it. All of these gaps represent the points in which security systems are at their weakest, hence porous to attacks.

The Need for a New Model

The primary reason why unknown attacks are difficult to detect is the data-driven nature of security systems. Most of them rely on information about known threats. They have to wait for updates on threat signatures for them to perform correct detection.

This does not mean that data-driven detection is wrong. The point is that it needs to be augmented with other strategies. As cybercriminals take advantage of artificial intelligence and machine learning to automate the production of new malicious software and other threats, the problem of unknown attacks is set to worsen in the absence of a paradigm shift.

Attackers can churn out many variants of existing threats, which are simply perceived as unknowns by most systems.

Security strategies should not be too focused on threat databases. Rather, it will be better to incorporate a threat-agnostic detection engine. Instead of totally relying on threat signatures, models of “clean execution flows” for applications in rendering certain files or links can be established.

These models mainly consist of whitelisted CPU-level code execution flows, which serve as benchmarks for what a normal or benign processing of files looks like. Files are then scanned and compared to the models of clean execution flows. If the way they are processed don’t coincide with existing clean execution models, the files are classified as threats, thus blocked or quarantined.

The Takeaway

Cybercriminals are relentless and will stop at nothing to succeed in their attacks. They know how to make new technologies work to their advantage, particularly the use of AI, to produce multitudes of variants of unknown threats – a major weakness of email security.

To stop them, it’s not enough to depend on data-dependent systems. Enterprises can benefit greatly from using model-driven approaches. Instead of relying on threat signature updates, security systems can examine how applications react/operate when faced with potentially malicious files.

Only those that match established models of normal operation are considered safe.