The MITRE ATT&CK framework refers to a globally accessible matrix of real-world adversary tactics and techniques. As its primary function, it has acted as a foundational knowledge base and standardised language that allows for more effective threat detection across the private sector, governments and cyber security companies. However, in recent years formal evaluations based on ATT&CK have led to more strategic product development and considerations in the EDR market.
A quick summary of the MITRE ATT&CK Enterprise matrix
ATT&CK stands for Adversarial Tactics, Techniques and Common Knowledge. While we will primarily be discussing the MITRE ATT&CK Enterprise framework (which looks at Windows, Linux and MacOS systems), the approach is also split into PRE-ATT&CK (pre-compromise) and Mobile ATT&CK matrices.
- Initial access
- Privilege escalation
- Defence evasion
- Credential access
- Lateral movement
- Command and control
Each tactic can be considered a step in the adversary’s overall mission and can be achieved by employing a variety of techniques. MITRE has collated hundreds of techniques that are known to be used in the wild; this list is based off of a database detailing clusters of adversary activity from over 107 actors and associations.
To know ATT&CK is to understand the enemy.
MITRE ATT&CK product evaluations
MITRE created Adversary Emulation Plans to allow red teams to actively simulate adversary behaviour and independently test their organization’s network security and security products against specific threats.
ATT&CK evaluation results can help you choose the right EDR
If you are looking for a convenient way to evaluate vendor results, MITRE have created a Vendor Comparison tool.
- What malicious activity the product can detect and alert you to;
- What relevant context was provided about the malicious activity; and occasionally
- How fast the product gathered relevant information about said activity.
The 2020 Enterprise evaluation is based on Carbanak and FIN7 emulations. Carbanak is a threat group that primarily targets banks, and FIN7 is a financially motivated group that has largely targeted the U.S. retail and hospitality sectors.
Interpreting MITRE ATT&CK results
While these evaluations do not provide the best vendor per se, it can answer the following questions for you:
- Does your product detect known threats to the organisation?
- How does your product present the data to your analysts?
When looking at the results, you need consider how the vendor detected the threat in each step. As in 2019’s evaluation, MITRE categorises detections into the following types:
- None: unable to collect any information related to malicious activity.
- Telemetry: minimally processed data that might indicate malicious activity – essentially an event log.
- MSSP: detection was based on human analysis.
- General: indicates anomalous or suspicious behaviour but does not provide further details on compromise.
- Tactic: information on potential intent of the activity provided.
- Technique: the product alerts you to what is going on and how the attack is conducted using the specific ATT&CK technique classifications.
While it is important that the product demonstrates they can effectively categorise alerts for analysts, it is equally important to note that not all malicious activity should fall under one specific detection type. It should be given the appropriate type for its behaviour.
However, a product that consistently produces a ‘none’ or ‘telemetry’ type throughout the assessment is likely to be low performing and could possibly lead to alert fatigue. Intrusion analysts and threat hunters cannot rely on EDR solutions that make them look for the supporting context of the threat.
As an additional resource, see MITRE’s ATT&CK evaluation results interpretation guide.
What are the limitations of these assessments?
The evaluation focuses entirely on detection – not prevention. The adversary behaviour is executed stage by stage and the product is barred from enacting preventative measures. This is because MITRE is solely evaluating detection performance, and this could not be done efficiently if the product were consistently shutting down attacks. Some solutions may deliver a variety of top-of-the-range prevention controls – it is necessary to do research outside of the assessment in order to fully evaluate those solutions as a whole.
It is also important to note that some attacks might require human expertise to fully uncover. You need to look at your existing tools and skillset, and from there decide which product capabilities are necessary for you.
Another issue to consider is how EDR vendors have now mapped their products to the framework, advertising that they ‘cover all MITRE listed attacks/techniques.’ However, during the evaluation the vendor might only encounter a few procedures simulated in a test environment. Therefore, this assessment provides an estimate of the solutions’ capabilities and should not be taken as the gold standard.
It is advantageous for any vendor producing detection tools to consult the unbiased and raw data held within the MITRE ATT&CK matrices, and moreover, let the evaluations drive their product enhancements. As MITRE continues to conduct these evaluations and update their database, the ATT&CK methodology (and products modelled off this framework) will grow more sophisticated. Governments, companies, and the cyber security industry will continue to reap the rewards of staying on top of known threats and will make more informed EDR investments and implementations.