In computer science, test coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs
A program with high test coverage, measured as a percentage, has had more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low test coverage
Basic coverage criteria
There are a number of coverage criteria, the main ones being:
Function coverage – Has each function (or subroutine) in the program been called?
Statement coverage – Has each statement in the program been executed?
Edge coverage – has every edge in the Control flow graph been executed?
Branch coverage – Has each branch (also called DD-path) of each control structure (such as in if and case statements) been executed? For example, given an if statement, have both the true and false branches been executed? Notice that this one is a subset of Edge coverage.
Condition coverage (or predicate coverage) – Has each Boolean sub-expression evaluated both to true and false?
https://en.wikipedia.org/wiki/Code_coverage
What is Code Coverage?
Code coverage is the percentage of code which is covered by automated tests. Code coverage measurement simply determines which statements in a body of code have been executed through a test run, and which statements have not. In general, a code coverage system collects information about the running program and then combines that with source information to generate a report on the test suite's code coverage.
Code coverage is part of a feedback loop in the development process. As tests are developed, code coverage highlights aspects of the code which may not be adequately tested and which require additional testing. This loop will continue until coverage meets some specified target.
Why Measure Code Coverage? It is well understood that unit testing improves the quality and predictability of your software releases
Measuring code coverage can keep your testing up to the standards you require.
Do you know, however, how well your unit tests actually test your code?
How many tests are enough?
Do you need more tests?
In summary, we measure code coverage for the following reasons:
To know how well our tests actually test our code
To know whether we have enough testing in place
To maintain the test quality over the lifecycle of a project
Use code coverage to determine how much code is being tested
To determine what proportion of your project's code is actually being tested by coded tests such as unit tests, you can use the code coverage feature of Visual Studio. To guard effectively against bugs, your tests should exercise or 'cover' a large proportion of your code
https://docs.microsoft.com/en-us/visualstudio/test/using-code-coverage-to-determine-how-much-code-is-being-tested?view=vs-2017
Code Coverage vs. Test Coverage: Pros and Cons
Code Coverage
This metric aims to measure number of lines covered by the test cases. It reports total number of lines in the code and number of lines executed by tests. Think of it as the degree to which the source code of a program is executed when a test suite runs. The intent is, the higher the code coverage, the lower the chance of having undetected software bugs.
Subtypes
Coverage is further split into many subtypes – function coverage, branch coverage, condition coverage, loop coverage, statement coverage and parameter value coverage.
Let me define the subtypes for the sake of clarity.
If tests cover all the function calls in code, function coverage is said to be 100%.
Similarly, if all branches in the code, i.e. all the if – else conditions have been tested with every possible input, then branch coverage is said to be 100%.
If all loop statements in the code are executed then loop coverage and statement coverage would be 100% respectively
For Python, a popular tool for code coverage measurement is Coverage.py
A measurement tool widely used for Go is gocov.
Test Coverage
Test Coverage aims to measure of the effectiveness of testing in a qualitative manner. It determines whether the test cases are covering entire functional requirements. You can think of it as a kind of black box testing, where test cases are not written based on code but based on user requirements or expected functionality.
Subtypes
The common mechanisms used for test coverage measurement are unit testing, functional testing, performance testing, integration or system testing and acceptance testing.
Unit tests are written at a granular level to check if a function/method performs as expected.
In functional testing, each functionality mentioned in the requirement document is tested.
Performance testing is generally a way to stress test the code for its responsiveness and stability at different workloads.
Integration testing or system testing is done to test if completely integrated product works in expected manner. Acceptance testing is generally done at the end of the development cycle. For acceptance testing, the product is handed over to the stakeholders and tested by them to determine whether the product is in acceptable state.
For Python, framework called PyUnit is available. JUnit, the well known Java testing framework serves as a foundation to launch tests on the Java Virtual Machine.
Go also offers a built in testing package for programs letting you easily write, organize and run tests
Code Coverage vs Test Coverage
So, now we know that code coverage is a measure of how much code is executed during testing, while test coverage is a measure of how much of the feature set is covered with tests.
Code Coverage Pros and Cons
Pros:
Helps capture bugs in program flow.
Cons:
Most Code coverage tools are relevant for Unit test only. It is essential to track all test types coverage (unit, API, System and even manual).
Test Coverage Pros and Cons
As a black-box testing methodologythis approach is aligned to the requirement specifications and has minimal interaction with code itself.
Cons:
Most of the tasks in such testing methodology are manual.
Function and feature are different. Covering a single feature will cover several functions, while the required effort may be similar.
Code coverage is a metric that can help you understand how much of your source is tested
Code coverage tools will use one or more criteria to determine how your code was exercised or not during the execution of your test suite.
Function coverage: how many of the functions defined have been called.
Statement coverage: how many of the statements in the program have been executed.
Branches coverage: how many of the branches of the control structures (if statements for instance) have been executed.
Condition coverage: how many of the boolean sub-expressions have been tested for a true and a false value.
Line coverage: how many of lines of source code have been tested
Code coverage tools can help you understand where you should focus your attention next, but they won't tell you if your existing tests are robust enough for unexpected behaviors.
Achieving great coverage is an excellent goal, but it should be paired with having a robust test suite that can ensure that individual classes are not brokenas well as verify the integrity of the system
When you've established your continuous integration (CI) workflow you can start failing the tests if you don't reach a high enough percentage of coverage. Of course, as we said it earlier, it would be unreasonable to set the failure threshold too high, and 90% coverage is likely to cause your build to fail a lot. If your goal is 80% coverage, you might consider setting a failure threshold at 70% as a safety net for your CI culture.
Once again, be careful to avoid sending the wrong message as pressuring your team to reach good coverage might lead to bad testing practices
https://www.atlassian.com/continuous-delivery/software-testing/code-coverage
Linting is the process of checking the source code for Programmatic as well as Stylistic errors. This is most helpful in identifying some common and uncommon mistakes that are made during coding
A Lint or a Linter is a program that supports linting (verifying code quality). They are available for most languages like JavaScript, CSS, HTML, Python, etc..
https://stackoverflow.com/questions/8503559/what-is-linting/8503586
A linter or lint refers to tools that analyze source code to flag programming errors, bugs, stylistic errors, and suspicious constructs.The term is originated from a Unix utility that examined C language source code.
https://en.wikipedia.org/wiki/Lint_(software)
EclEmma
EclEmma is an Eclipse plug-in that generates code coverage reports and provide simple trace information about test cases Coverage is defined as a measure of the completeness of the set of test cases.
This definition means that the more source code that is executed by your test cases the better.
There are four types of coverage:
Method coverage: how many of your methods have been called by your test cases?
Statement coverage: how many of your statements have been run by your test cases?
Decision/Branch coverage: how many decision points (if statements) have evaluated both true and false?
Condition coverage: how many Boolean sub-expressions have been evaluated both true and false at a decision point? EclEmma may be used on top of JUnit. EclEmma runs the JUnit test cases and generates the coverage report from the execution. EclEmma provides overall, package, and file level information on the statement.
http://agile.csc.ncsu.edu/SEMaterials/tutorials/eclemma/
Coverage.py
Coverage measurement is typically used to gauge the effectiveness of tests. It can show which parts of your code are being exercised by tests, and which are not.
https://coverage.readthedocs.io/en/v4.5.x/
gocov
Coverage reporting tool for The Go Programming Language
https://github.com/axw/gocov
EMMA
EMMA is a fast Java code coverage tool based on bytecode instrumentation. It differs from the existing tools by enabling coverage profiling on large scale enterprise software projects with simultaneous emphasis on fast individual development.
http://sourceforge.net/projects/emma/
Crap4j
Crap4j report screenshot. Crap4j is a Java implementation of the CRAP (Change Risk Analysis and Predictions) software metric. The CRAP metric combines cyclomatic complexity and code coverage from automated tests (e.g. JUnit tests) to help you identify code that might be particularly difficult to understand, test, or maintain
www.crap4j.org
Cobertura
Cobertura is a Java code coverage reporting tool. It is based on jcoverage and works on any platform with Java 5 or higher. You can use Cobertura with ant, maven or command line access. Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage. http://cobertura.github.io/cobertura/
djUnit
djUnit uses jcoverage to generate a coverage report of the performed tests, and it can be checked on Eclipse. The results are shown on djUnit coverage view , task list and java source editor. http://works.dgic.co.jp/djunit/
Lint4j
Lint4j ("Lint for Java") is a static Java source and byte code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis.
http://www.jutils.com/
The Sqrrl Security Analytics Company provides a broad framework for conducting cyber threat hunt operations
1.Create a Hypothesis
2.Investigate via Tools and Techniques
3.Uncover new Patterns and
Tactics, Techniques, and Procedures (TTPs)
4.Inform and Enrich Analytics
These steps describe the essence of conducting cyber threat hunt operations;
however, specific details such as planning, implementation, and specific TTPs are left to
the organization to determine.
3.3 Creating a Hypothesis
A foundation built on threat intelligence transitions into a defining characteristic of threat hunting: a hypothesis. Simply speaking, threat hunting starts with a question:
What threats may be in the organization? How would the adversary infiltrate the organization? What would their objectives be? These questions enable Threat Hunters to
develop a specific and measurable hypothesis grounded in an understanding of the threat
which will ultimately drive the hunt operation.
For a hypothesis to be effective, it has to be testable. An example hypothesis
could state that executive management is at an elevated risk of compromise from state
-sponsored actors who seek trade secret information. Threat hunters could then evaluate possible threat vectors such as spear phishing
https://www.sans.org/reading-room/whitepapers/detection/scalable-methods-conducting-cyber-threat-hunt-operations-37090
SOC teams don’t have expertise to build high quality rules
“Out of the box” rules don’t work; they end up generating too many false positives
They don’t have the bandwidth to go threat hunting
Threat hunting is proactively searching for indications of any IT security threat or compromise. It is about filling in the gaps in the SIEM rule set. It’s accounting for the false negatives—the lack of alerts in situations when alerts really should be raised—in rule-based security systems.
To find new threats, SOC teams need to analyze security events, billions of which occur every day in a large enterprise.
Instead of sampling security, threat hunting relies on scoring. Security analysts assign each factor a numerical score that estimates the factor’s importance
For example, let’s say a file is created on a server. The name of the file might be quite conventional, earning that file-name factor a rating of 1. The fact thatthe file was created by a superuser might earn the file-creator factor a rating of 10. Because the file was created at 3 am on a Saturday morning when the data center was empty, analysts might decide to assign the factor of the time of file creation a rating of 10, as well.
It’s through this detailed scoring of factors associated with events that threat hunters can identify anomalous events that bear closer inspection Scores of factors are combined, and the high-scoring outliers clearly stand out, signaling the possibility of a threat.
They search for high-scoring anomalous events, and if they find any, they examine them closely to determine if they indicate the presence of a genuine threat.
You can train software to look for anomalies in events, and to examine them for indications of threats.
If the automated threat hunting is based on machine learning, you can train the software to accept and apply corrections and refinements from security analysts, so that the automated threat hunting becomes increasingly fast and accurate over time.
https://www.csoonline.com/article/3254230/cyber-attacks-espionage/how-to-automate-threat-hunting.html
The difference between Rule-Based Approach and Threat Hunting can be understood in simple form as
Rules: Watchman (Gate) or Traps (Multiple Locations) Threat Hunting: Patrolling or Raids
https://securitycommunity.tcs.com/infosecsoapbox/articles/2018/05/22/cyber-threat-hunting-%E2%80%93-overview-and-techniques
We define hunting as the process of proactively and iteratively searching through networks to detect and isolate advanced threats that evade existing security solutions
There are a number of factors to consider when judging an organization’s hunting ability, including:
»The quantity and quality of the data they collect;
»In what ways they can visualize and analyze various types of data;
»What kinds of automated analytic they can apply to data to enhance analyst insight
https://sqrrl.com/media/Framework-for-Threat-Hunting-Whitepaper.pdf
Definition of a Cyber Threat Hunter
Cyber threat hunters are information security professionals who proactively and iteratively detect, isolate, and neutralize advanced threats that evade automated security solutions. Cyber threat hunters constitute an integral part of the rapidly growing cyber threat intelligence industry. Their employment gives companies a competitive edge as opposed to using traditional rule or signature-based detection methods.
Responsibilities of a Cyber Threat Hunter
Security experts use the 80/20 principle to assess cyber threats. Eighty percent (80%) of cyber threats are unsophisticated and can be mitigated with good security hygiene, while the remaining twenty percent (20%) tend to be more advanced threats. Still, about half of these advanced attacks can be successfully addressed with different blocking and tackling techniques.
The other half of advanced attacks constitutes the top 10% of cyber threats. These highly advanced threats cannot be detected solely with programmatic solutions. Cyber threat hunters aim to sniff out these highly advanced cyber threats. Their job is to track and neutralize adversaries who cannot be caught with other methods. The threats they hunt for can be posed by either an insider, such as an employee of the organization, or an outsider, such as an organized crime group
Cyber Threat Hunter Tools and Techniques
Instead of trying to infiltrate the environment from the outside as it happens during penetration testing, cyber threat hunters work with the assumption that adversaries are already in the system.
● Security Monitoring Tools - Cyber threat hunters work with all kinds of security monitoring solutions such as firewalls, antivirus software, network security monitoring, data loss prevention, network intrusion detection, insider threat detection, and other security tools. Besides monitoring the network at the organizational-level, they also examine endpoint data. They gather event logs from as many places as possible, as their work requires a sufficient amount of security data.
● SIEM Solutions - Security Information and Event Management (SIEM) solutions gather internal structured data within the environment and provide a real-time analysis of security alerts from within the network. Basically, they turn raw security data into meaningful analysis. SIEM tools don’t only help manage the huge amount of data logs hunter work with, but also make it possible to find correlations which can reveal hidden security threats.
● Analytics Tools - Cyber threat hunters work with two kinds of analytics tools: statistical and intelligence analysis software. Statistical analysis tools, such as SAS programs, use mathematical patterns instead of pre-defined rules to find odd behavior and anomalies in the data. Intelligence analytics software visualizes relational data and provide security professionals with interactive graphs, charts, and other data illustrations. They make it possible to discover hidden connections and correlations between different entities and properties in the environmen
Correlation and Regression are the two analysis based on multivariate distribution. A multivariate distribution is described as a distribution of multiple variables. Correlation is described as the analysis which lets us know the association or the absence of the relationship between two variables ‘x’ and ‘y’. On the other end, Regression analysis, predicts the value of the dependent variable based on the known value of the independent variable, assuming that average mathematical relationship between two or more variables.
https://keydifferences.com/difference-between-correlation-and-regression.html
Rule-based threat detection approach has many limitations (like static nature i.e. detection based on a pre-defined threshold value, inability to process data in high throughput scenario, the requirement to exponentially increase computing resources as the number of rules increases etc.) and threat hunting effectively compliments the rule-based approach by addressing the above-mentioned limitation
This model emphasizes the relationships and characteristics of four basic components: the adversary, capabilities, infrastructure, and victims.
The main axiom of this models states, “For every intrusion event, there exists an adversary taking a step toward an intended goal by using a capability over infrastructure against a victim to produce a result.
This means that an intrusion event is defined as how the attacker demonstrates and uses certain capabilities and techniques over infrastructure against a target
https://www.recordedfuture.com/diamond-model-intrusion-analysis/
The Network-Centric Incident Response and Forensics Imperativev1.0
Traditional Host-Centric IR and Forensics
Standard host-centric incident response and forensics scenario assumes:–Investigators know what systems are suspected of being compromised–Live response will yield reliable results that can be interpreted–Forensic duplication of a hard drive will show evidence of compromise
Network security monitoring is the collection, analysis and escalation of indications and warning to detect and respond to intrusions
NSM relies upon four forms of traffic-centric data
Statistical data(Capinfos, Tcpdstat, Trafshow)•
Descriptive, high-level view of aggregated events–
Session data(Argus, SANCP, NetFlow)•
Summaries of conversations between systems•
Content-neutral, compact; encryption no problem–
Full content data(Tcpdump, Tethereal, Snort as packet logger)•
All packet details, including application layer•
Expensive to save, but always most granular analysis–
Alert data(Snort, Bro, other IDSs)•
Traditional IDS alerts or judgments (“RPC call!”)•
Context-sensitive, either by signature or anomaly•
Sguil(www.sguil.net) is an interface to much of this in a single open source suite
NSM is not SIM/SEM: a SIM/SEM collects and correlates log sources which may or may not have any value
Network-Centric IR and Forensics
Identify the scope of the intrusion
1. Deploy sensor with Sguil, or in a pinch, Tethereal and Argus
2. Conduct traffic threat assessment using session data to discover anomalous connections
3. Validate anomalous connections using full content data
4. Evidence of suspicious or malicious connections points me to potential victims
5. Review all connections to or from potential victims
6. Resulting systems receive host-based live response
7. Any time evidence reveals high probability of compromise, I recommend disabling access to victim until further course of action decided upon by client
8. Recommend hard drive forensics
9. As intruder's modus operandi is learned, deploy custom Snort signatures to provide alert data
“Threat hunting is the process of seeking out adversaries before they can successfully execute an attack.”
6. What is the difference between threat hunting and other prevention- and detection-based methods?
threat hunting is very much a proactive security methodology that makes use of sophisticated analytical tools such as artificial intelligence and machine learning. The threat-hunting process starts with formulating a specific hypothesis in which the catalyst for this was some kind of alert, assessment or even the results of a penetration test. This hypothesis will then be tested by using the abovementioned tools to search for this potential cyberthreat.
False negative vs. false positive: how can Next-generation SIEM help?
False negative vs. false positive
A false negative is a security incident that was not detected in a timely manner. For example, a phishing attack resulting in a compromised user account that goes unnoticed by the security team until more damage occurs.
A false positive, on the other hand, is an alarm generated by security systems that indicates a security incident has likely occurred when, in fact, everything is normal.
Realistically, organisations that want to reduce false negative risk will need to accept increased false positive frequency and staff their security operations centre appropriately.
Unfortunately, some vendors sell AI and Machine Learning (ML)-based behavioural anomaly detection as an easy button for advanced threat detection and false positive reduction.
osquery exposes an operating system as a high-performance relational database. This allows you to write SQL-based queries to explore operating system data. With osquery, SQL tables represent abstract concepts such as running processes, loaded kernel modules, open network connections, browser plugins, hardware events or file hashes.
Osquery uses basic SQL commands to leverage a relational data-model to describe a device.
Performant endpoint visibility
https://osquery.io/
Cyber threat hunting is a proactive security search through networks, endpoints, and datasets to hunt malicious, suspicious, or risky activities that have evaded detection by existing tools.
there is a distinction between cyber threat detection versus cyber threat hunting
Threat detection is a somewhat passive approach to monitoring data and systems for potential security issues, but it’s still a necessity and can aid a threat hunter
Rather than sit back and wait for threats to strike, cyber threat hunting develops hypotheses based on knowing the behaviors of threat actors and validating those hypotheses through active searches in the environment
With threat hunting, an expert doesn't start from an alert or even Indicators of Compromise (IOC) but deeper reasoning and forensics.
In many cases the hunter’s efforts create and substantiate the alert or IOC.
Cyber threat hunting aggressively assumes that a breach in the enterprise has or will occur.
AWS serverless system to analyze and response to threat logs Palo Alto Networks
So when an user download a file, Prisma Access passes the file to user while let Wildfire scan the file for threat verdict, then Primsa will get signature to block the file later if it is identified as malware.
The Relay itself is just a simple application written in Python that can be easily packaged and deployed. This relay is now Cisco Hosted and no longer requires AWS Lambda
threat intel aggregation platforms which ingest the same data feeds
a central, scalable and easily queryable repository for threat intelligence of all types
Utilizes amazon services to allow for minimal support needs while maintaining scalability and resilience and performance. (aws lambda, elasticsearch, s3, sns)
Serverless - No maintenance required
Scalable (all services scale via AWS)
High performance API - API can be used to run extremely high volume queries
Flexible - Feeds can be added via simple json feed configuration
Extensible - written in python and extended by new modules
Cost-effective - Pay only for the backend services - don't worry about API limits
Automated Deployment - platform can be deployed from a single command
Works "out of the box" - comes pre-configured with 30+ opensource intel feeds
https://github.com/securityclippy/elasticintel
gather all IOCs for Talos Intelligence's weekly threat roundup and post to MISP for storage.
The ThreatMapper console can be deployed on a single docker host or in a Kubernetes cluster.
ThreatMapper then monitors your development or production workloads using Sensor Agents. The sensors can be deployed on a wide range of platforms - Kubernetes, Docker, Fargate, Bare-Metal and Virtual Machines
https://github.com/deepfence/ThreatMapper
where the C2 was TLS encrypted, as most traffic is these days- but the C2 server was actually “serverless” code running in the Azure cloud. In this scenario, what is seen on the network is an encrypted tunnel to a subdomain of azurewebsites.net. In many networks, there can be thousands of unique sessions per day, to hundreds of subdomains of azurewebsites.net
The malware in question persists as an Office add-in, which complicates detection on both the network and endpoint itself.
This malware only runs when certain Office applications are started (which could be quite frequent based on the victim)
And because it runs inside the Office process, it makes it more difficult to detect on the
endpoint
it’s possible to load malicious add-ins without elevated permissions, user-intervention or notification, and malicious add-ins can download and run other executables without user knowledge with normal user-level permissions
Take a malicious DLL
Put it in a directory that unprivileged users have access to (%appdata%\Roaming\Microsoft\Word\startup\)
Change the extension from .dll to .wll
Word will run the code in the .dll every time the application is opened. Similar capabilities exist for Excel, PowerPoint, etc.
It doesn’t matter even if you have configured Word to disable add-ins since that setting does indeed disable add-ins, except WLLs.
If the malicious .dll is totally self-contained (meaning it doesn’t need to download and run other files), then examining the process will show nothing because all the code is running in Word.
The process when the add-in does not execute other applications. The code runs within WINWORD.exe.
The attackers, in this case, created an add-in that is just a simple loader and clean-up for other executables
DLL’s with simple functionality are usually more difficult to statically detect as malicious because they don’t have many overtly suspicious-looking features.
Under the WINWORD.EXE process is a child process with the same name (WINWORD.exe), however, the child WINWORD is not signed by Microsoft.
The executable downloaded and executed from the Internet is shown as a child process of WINWORD.exe.
This file was also named WINWORD.exe, but seeing conhost.exe tells us the second WINWORD is not a GUI application
It is a console application and is hidden from the user because it ultimately launched in a hidden state by ShellExecute.
the child WINWORD process is not signed by Microsoft. This is the malicious second-stage binary.
One disadvantage to a completely self-contained .dll is that static detection is easier since the single .dll will likely contain a high percentage of suspicious functionality.
Serverless C2
if you wanted to set up a server, you needed to find someplace to put it
But you still need to manage an entire server instance simply to run some simple task (like C2)
most C2/botnet controllers were relatively simple PHP applications, but the attacker still needed to manage a full server only when a relatively simple task needed to be accomplished
Serverless computing has eliminated all that headache. Now, you can take your web app code (such as that PHP-based C2), upload the code to a serverless function provider, and it just works without needing to worry about the underlying system running the app
Since functions in code usually take some input and give some output after running, the same happens with serverless functions.
The difference is you pass parameters and data into a serverless “function” with a GET or POST request and receive the results in a web response.
This is terrifying from a threat detection and hunting perspective because the vast majority of a company’s Internet traffic is already going to Microsoft, Google, Amazon, and Cloudflare – and all of it is pretty much encrypted, too.
what happens when Microsoft Word is started up?
Word starting up with no add-ins present
Several [a-z]-ring.msedge.net connections
Possible ocws.officeapps.live.com connections
Finally fp.msedge.net
you’ll see the following if an add-in is loaded that communicates on the network:
Word starting up, with the malicious add-in present. It downloads and executes the second stage payload from an azurewebsites.net function.
Several <a single letter>-ring.msedge.net connections
<request(s) from add-in(s)>
Possible ocws.officeapps.live.com connections
Finally fp.msedge.net
the serverless C2 is caught because of the surrounding contextual sessions seen when Word starts up
Identify the “network fingerprint” of Word starting up
Then identify the additional add-in connections
Alert to “add-in connections that are not common”
But what happens when we have serverless C2 malware that doesn’t run as an add-in and running instead as a scheduled task once-a-day so there aren’t meaningful surrounding sessions to analyze?
a detection recipe that can alert if traffic is seen:
To a destination that is rare for the enterprise, and
That activity is seen [in the case pictured above] at least five of the past 14 days.
or even change the recipe itself to represent almost any attacker TTP – like “alert when a device connects directly to an IP address (no DNS request) for 3 of the past 7 days.
Threat hunting is an active IT security exercise with the intent of finding and rooting out cyber attacks that have penetrated your environment without raising any alarms. This is in contrast to traditional cybersecurity investigations and responses, which stem from system alerts, and occur after potentially malicious activity has been detected
Threat hunting involves going beyond what you already know or have been alerted to.
Who should be involved in threat hunting?
To carry out a threat hunting campaign, a mix of core skills is needed in a team. These skills include:
Familiarity with endpoint and network security. You will need seasoned members of your SOC
Understanding of data analytics. An understanding of statistical analysis will help to identify patterns in the data.
Innate curiosity. Threat hunting can sometimes be likened to an artistic pursuit. It requires a certain amount of creative thinking to connect seemingly unrelated items or ask, "I wonder what would happen if…"
When should you do threat hunting?
You may wish to undertake a threat hunting exercise when you suspect risky behavior has occurred
You need to set a scope for the hunt, identify clear goals, and set aside a block of time to perform the exercise.
When you are done, you need to assess steps to improve your security posture, establishing threat prevention playbooks to address the results moving forward.
Where should you hunt for threats?
Ultimately, data is key to any successful threat hunt.
Before you can do anything related to threat hunting, you need to ensure you have adequate logging capability to carry out the hunt. If you can't see what is happening on your systems, then you can't respond in kind. Choosing which systems to pull data from will often depend on the scope of the hunt.
In general, cybersecurity strategies focused on threat detection attempt to identify an incoming or ongoing attack and then prevent or quickly remediate it. However, this approach has its issues as it assumes that all attacks can be detected and mitigated before any damage is done.
Cyber threat hunting involves proactively searching for unknown vulnerabilities and undetected attacks within an organization’s environment. Based on cyber threat intelligence, known attack techniques, and other information, threat hunters develop and test hypotheses about potential threats by collecting and analyzing data from various sources inside and outside of the organization.
Cyber threat hunting is designed to allow an organization to detect and respond to potential threats that it does not know exist and has not detected via other means. This provides the organization with more comprehensive protection against cyber threats and the ability to detect and mitigate attacks and security gaps that its existing security architecture has missed
How to Perform a Threat Hunt
Accomplishing this requires using a threat hunting framework to plan threat hunting exercises, such as this five-step process.
Step 1 – Hypothesis
Threat hunting is designed to identify an unknown threat to an organization’s cybersecurity
Without a known attack or a particular threat to investigate, threat hunters need a starting point for their investigations
This could be a potential vulnerability in the company’s systems or the tactics, techniques, and procedures (TTPs) of a known threat actor.
Based on this hypothesis, the threat hunter can then develop a strategy for identifying whether the suspected threat is present on the company’s systems.
Step 2 – Collect and Process Intelligence and Data
With a plan in place, the threat hunter will collect and process the data required to prove or disprove their hypothesis.
Collecting and analyzing data from internal and external sources often requires specialized tools, such as security information and event management (SIEM) and dark web monitoring solutions.
Step 3 – Trigger
After collecting and analyzing the required data, the threat hunter should be able to determine whether or not the hypothesis is correct
After proving the existence of the threat, the hunter is ‘triggered’ to perform an in-depth investigation to determine the scope and details of the incident required for remediation.
Step 4 – Investigation
By identifying infected systems and determining details about how the attack was performed and its impacts, the threat hunter can determine what remediation steps are necessary
In addition to searching corporate systems for signs of infection, threat hunters should look on dark web marketplaces for stolen data or other information about the attack.
Step 5 – Resolution
By the end of their investigation, the threat hunter should have a complete picture of how the attack was carried out, its objectives, and the impacts on the organization and its system
This information should inform the actions that the organization takes to remediate the incident
Threat Hunting Methodologies
Adversary Hunting
Adversary hunting involves searching for indications that a particular threat exists on an organization’s systems
Different threat actors have known TTPs that they use across different attack campaigns. For example, a particular APT may be known for exploiting VPN vulnerabilities, so a threat hunt may be focused on identifying if VPNs are vulnerable or if unusual activity has been detected on VPN endpoints.
Threat hunters can develop a hypothesis that a particular threat actor is using one of their known TTPs within an enterprise environment. After performing this hunt, the exercise can be repeated for other known TTPs for the threat actor or for other potential threats
Hypothesis-Based Hunting
All threat hunting is based on developing a hypothesis and testing it.
This threat hunting method focuses on using certain methods to create these hypothese
Data Analytics: Machine learning (ML) algorithms can analyze large volumes of security data and extract trends and anomalies from them. The results of these analytics can be used to develop hypotheses for use in threat hunting.
Threat Intelligence: Companies can collect threat intelligence from a variety of internal and external sources such as malware analysis, dark web monitoring, and vulnerability scans. This threat intelligence can be used to identify probable threats that can be investigated via threat hunting.
Risk Assessment: A corporate risk management program should identify an organization’s most valuable assets and the greatest threats to them. This information can be used to focus threat hunting on investigating the potential threats that pose the greatest risk to the organization.
Investigation Using Indicators of Attack
Some resources, such as MITRE’s ATT&CK framework, provide in-depth information about the ways that an attacker can achieve a particular objective and how these techniques can be detected and mitigated
Tools like MITRE ATT&CK can be used as a framework for developing a threat hunting strategy.
By searching for the different indicators of attack and compromise outlined within the ATT&CK frameworks, threat hunters can determine if their organization has been targeted by attackers using these techniques
Why Should You Utilize Threat Hunting?
Most organizations have a detection-focused security strategy;
however, this is a reactive approach to managing cyber risk.
Threat hunting is a proactive activity that complements threat detection
Detecting Intrusions: Proactive threat hunting is invaluable because it enables organizations to identify threats that were performed without being caught by existing defenses.
Identifying Vulnerabilities:Threat hunting can help with detecting and remediating previously unknown vulnerabilities within an organization’s systems.
Quantify Risks:Threat hunting can help to inform risk analysis by determining the company’s vulnerability to various cyber threats
Improving Defenses:Threat hunting can help to identify detection gaps and develop strategies for building visibility into additional cyber threats
Streamline Threat Detection: Threat hunters may identify more efficient ways to collect and analyze data to detect various threats, enabling them to streamline threat detection and eliminate the collection of unnecessary data.
Threat Hunting Best Practices
only effective if the threat hunting program is designed and implemented properly.
Define a Dedicated Threat Hunting Team
Threat hunting may seem less important because it focuses on hypotheticals rather than responding to known threats to the organization
However, these proactive investigations are vital to detecting more sophisticated and unknown threats.
Defining a dedicated threat hunting role or a minimum number of hours to spend threat hunting each week is essential to ensuring that threat hunting is actually performed.
Develop the Right Skill Sets
Threat hunters need to know how to develop and test hypotheses about potential threats to the organization
They also need in-depth knowledge and experience with the various platforms within an organization’s environment to perform these tests.
Acquire Specialized Hunting Tools
Efficient and effective threat hunting requires the ability to rapidly prove or disprove hypotheses about threats to the organization
Investing in certain security solutions – such as a SIEM and dark web monitoring solution – can help to dramatically expedite the threat hunting process.
Prioritize Based on Risk
When planning threat hunting investigations, it is important to prioritize them based on the potential risk to the organization
Different risks have different levels of probability and potential impact on the organization
Focusing on probable and high-risk threats helps to maximize the benefit of the threat hunt to the organization
Automate When Possible
The vast amount of data that threat hunters need to collect, aggregate, and process makes automation an invaluable tool
In addition to data collection, automation can also help with developing hypotheses and focusing the attention of threat hunters. For example, the use of artificial intelligence (AI) and user and entity behavior analytics (UEBA) can help with identifying abnormal events that deserve investigation.
What Are You Hunting For?
Indicators of Compromise (IOCs): IOCs are data regarding a past security incident. This includes log files, forensic data, and similar information.
Indicators of Attack (IOAs): IOAs are information about an ongoing attack. These are similar to IOAs but require real-time or near real-time access.
Network Artifacts: Monitoring network traffic can help detect cyberattacks by looking for malware command and control (C2) traffic, attempted exploits of vulnerabilities, etc.
Host Artifacts: Malware infections and other cybersecurity incidents can create artifacts on endpoints such as files, processes, registry entries, and more.
Adversaries: Based on knowledge of threat actors’ motivations and TTPs, threat hunters can look for signs of their presence within an organization’s environment.
Which Threat Hunting Platform Types Are There?
Threat hunters need to be able to search through large amounts of data to identify trends and determine whether or not certain events have occurred
three of the most important types of threat hunting platforms include:
Security Monitoring Tools: Threat hunters need security data to investigate and evaluate their hypotheses. Security monitoring tools like firewalls, antivirus, and similar solutions generate and collect this data
SIEM Solutions:SIEMs automatically collect and aggregate this data into a single platform, making it easy to view and analyze
Analytics Tools:Data analytics tools help threat hunters to extract trends and outliers from their datasets.
Threat hunting is the practice of proactively searching for cyber threats that are lurking undetected in a network.
After sneaking in, an attacker can stealthily remain in a network for months as they quietly collect data, look for confidential material, or obtain login credentials that will allow them to move laterally across the environment
Once an adversary is successful in evading detection and an attack has penetrated an organization’s defenses, many organizations lack the advanced detection capabilities needed to stop the advanced persistent threats from remaining in the network. That’s why threat hunting is an essential component of any defense strategy.
Threat Hunting Methodologies
Threat hunters assume that adversaries are already in the system, and they initiate investigation to find unusual behavior that may indicate the presence of malicious activity
this initiation of investigation typically falls into three main categories:
1. Hypothesis-driven investigation
Once a new TTP has been identified, threat hunters will then look to discover if the attacker’s specific behaviors are found in their own environment.
2. Investigation based on known Indicators of Compromise or Indicators of Attack
This approach to threat hunting involves leveraging tactical threat intelligence to catalog known IOCs and IOAs associated with new threats. These then become triggers that threat hunters use to uncover potential hidden attacks or ongoing malicious activity.
3. Advanced analytics and machine learning investigations
The third approach combines powerful data analysis and machine learning to sift through a massive amount of information in order to detect irregularities that may suggest potential malicious activity. These anomalies become hunting leads
Threat Hunting Steps
Step 1: The Trigger
Often, a hypothesis about a new threat can be the trigger for proactive hunting. For example, a security team may search for advanced threats that use tools like fileless malware to evade existing defenses.
Step 2: Investigation
During the investigation phase, the threat hunter uses technology such as EDR (Endpoint Detection and Response) to take a deep dive into potential malicious compromise of a system
Step 3: Resolution
The resolution phase involves communicating relevant malicious activity intelligence to operations and security teams so they can respond to the incident and mitigate threats.
The data gathered about both malicious and benign activity can be fed into automated technology to improve its effectiveness without further human intervention.
AWS CloudWatchs vs. CloudTrail - What's the Difference?
AWS CloudWatch is a monitoring service. That means it allows you to monitor the performance of your AWS resources and applications
analyze logs
when a resource/application fails, determine what happened and why by looking at the logs
monitor applications
monitor EC2 metrics such as CPU utilization, memory used, status check, network throughput, and more
optimize resources
specify what happens when a specific threshold is met or not.
AWS CloudTrail
What action was taken
Who performed it
When the action was taken
Where the action was taken
For instance, let’s say your S3 bucket was deleted by mistake. You can use AWS CloudTrail to see who deleted the bucket, when, and where (e.g. API Call or from the AWS Management console)
AWS provides another
service called Amazon
Athena, which runs SQL
queries against data in
an Amazon S3 bucket
Amazon GuardDuty is a managed service that is evaluating a growing number of findings
that detect adversary behaviors and alerting the customer. Amazon GuardDuty evaluates
potential behaviors by analyzing Amazon VPC Flow Logs
real-time VPC flow
logs analysis engine can be created using AWS Lambda, Amazon Kinesis, Amazon S3,
Amazon Athena and Amazon QuickSight.
SIEMs in the Cloud
The threat hunting team should focus on developing and managing a tactical SIEM,
which could be different from the SIEM a SOC might use.
The tactical SIEM will likely
have unstructured data, a shorter retention policy than the SOC’s SIEM, and the ability
to easily determine what the infrastructure looked like in the recent past.
After the tactical SIEM is stood up; the data is gathered, translated and enriched; and
mechanisms for analytics and reporting are in place, the threat hunting team will start
to discover repeated steps, analytics or actions. An emerging service that integrates
with the SIEM, called Security Orchestration, Automation and Response (SOAR), can be
helpful there.
The company behind Elasticsearch, Elastic,
has released a new app called the Elastic SIEM that is more focused on the security
operations. Other products, such as ones from Sumo Logic and Splunk, also integrate
directly with AWS and provide even richer and more full-featured analytic platforms
Soaring with SOAR
Threat hunting is all about proactive analysis of data to detect the anomalous behavior
that is undetectable by the security products.
As the threat hunting team’s analytics
become more sophisticated, it may begin developing a set of repeatable analytics,
enrichments or data gathering steps
A SOAR leverages the data storage and enrichment of the SIEM, understands
basic rules of infrastructure integration and allows the easy buildout of playbooks to
automate a course of action
In the web application use case, if there are several failed SQL injection attempts, the
final attempt could signify the last failure before success.
The process of information
from that host at that time would be of interest.
SOAR could be used to identify that
ultimate SQL injection failure, tag it and then also tag the process log information from
that time.
The next step in the playbook could be to move those logs into a separate
Amazon S3 bucket for more accessible analysis.
The process logs by themselves could
then be enriched by validating with a malware signature API to identify whether the
process is known good or not.
Gathering potential logs to analyze and automating the
enriching processes when necessary could save threat hunters tedious and repetitive
work. It could also help provide quicker triage. The SIEM with a SOAR could significantly
improve speed to analysis.
it’s possible to use data pushed to the SIEM and
SOAR, such as the SQL injection detection logs from the WAF, and initiate an action
Rather than always pull the process list on an hourly basis, the SIEM could execute
host-based tools, such as OSQuery, to reach out to the suspect web server and pull the
process list in near real time.
This automated response action allows the team to limit
what passive data has to be managed, and makes it easier to correlate the process logs
returned with the suspicious SQL injection attacks.
In the Amazon EC2 use case, the SIEM/SOAR could review the READs from an EC2 to an
Amazon S3 bucket and detect a first-time READ to an S3 bucket.
The SOAR playbook executes a host agent such as OSQuery to interact directly with that EC2 to pull fresh process
information and kick off a scan with Amazon Inspector.
OR
The SOAR playbook uses AWS services such as Amazon Inspector and AWS Systems Manager to interact directly with that EC2 to pull fresh process
information and kick off a scan with Amazon Inspector.
It then gathers all these reports
and provides them in a single artifact bucket for the security analysts, creating a high-
priority message in the corporate chat system or sending out SMS alerts to on-call
personnel
Some of the more sophisticated SOARs, such as Palo Alto’s Demisto and Splunk’s
Phantom, also allow for the detection of cascading anomaly triggers that can perform
automated remediations—taking our use cases together to build a sophisticated
SOAR playbook.
SOAR Playbook Use Case
The attacker performs several SQL injection attacks against a particular EC2.
The SOAR
kicks off a process listing and tags all logs from that EC2 with a unique identifier.
One of
those logs with the unique identifier specifies a failed Amazon S3 bucket listing attempt
The
SOAR identifies that this failed bucket listing happened on an EC2 that is being triaged
Because the organization is using auto-scaling, the SOAR notifies the auto-scaling
system to deregister the EC2 (i.e., pull that EC2 out of service but keep it running)
The
SOAR playbook waits for the deregistering to finish, then removes all security groups
except triage, and the triage group effectively isolates the EC2 from all other systems
The SOAR then performs a memory dump of the EC2, takes a snapshot and stops the
EC2.
All the data is gathered up and prepared in an Amazon S3 bucket for the security
team when it is ready to investigate.
Organizations are moving away from traditional server-based infrastructure into
serverless, event-driven architectures that rely on native cloud services
Threat hunting is critical to finding the advanced attacker techniques that have escaped
the detection of deployed security products.
Proper strategy ensures the right data is collected, enriched and available to the tools
the threat hunting team uses to tease out suspicious anomalies from the vast and ever-
Without the right data, you cannot hunt. Threat hunting requires first collecting quality data from various sources, such as logs, servers, network devices, firewalls, databases, and endpoints
Effective threat hunting requires more than just visibility though. During the investigation, threat hunters need to leverage the right tools in order to establish a baseline and proactively investigate anomalies.
To catch a criminal, you need to think like a criminal. You must assume a breach will happen and look at the problem from the attacker’s point of view. Threat modeling involves identifying potential threats and modeling avenues of attack
