Monday, May 27, 2019

Containers and Security


  • Let’s go over some of the things you can do with containers that you CANNOT do with Jails or Zones or VMs.
You can have your application running in one container, then in a different container sharing a net namespace you can run wireshark and inspect the packets from the first container.
You could also do the same with sharing a pid namespace, except instead of running wireshark you can run strace and debug your application from an entirely different container

This extra complexity leads to bugs that lead to container escapes. Don’t get me wrong you could also escape a VM, Jail or Zone, but the design is not as complicated as that of the primitives that make up containers.
You can get a sandbox level of isolation with containers, from your pieces of Seccomp, AppArmor, and SELinux profiles.

https://blog.jessfraz.com/post/containers-zones-jails-vms/
  • We have derived four generalized use cases that should cover security requirements within the host-container threat landscape. The use cases include: (I) protecting a container from applications inside it, (II) inter-container protection, (III) protecting the host from containers, and (IV) protecting containers from a malicious or semi-honest host. We found that the first three use cases utilize a software-based solutions that mainly rely on Linux kernel features (e.g., namespaces, CGroups, capabilities, and seccomp) and Linux security modules (e.g., AppArmor). The last use case relies on hardware-based solutions such as trusted platform modules (TPMs) and trusted platform support (e.g., Intel SGX).

1) Use Case I: Protecting a Container From Applications Inside IT
In this use case, each application within a running container ci∈C can be honest, semi-honest, or malicious. We assume that applications cannot break access control policies if set. Additionally, we assume that some applications might require root privileges (or parts of the full root access). 

2) Use Case II: Inter-Container Protection
In this use case, we assume that one or more of the containers are semi-honest or malicious
These containers can be inside the host H or on different hosts. Being on different hosts is important for the emerging field of service meshes

3) Use Case III: Protecting the Host (And the Applications Inside it) From Containers.In this use case, we assume that at least one container is semi-honest or malicious within the host.A semi-honest container can have access to confidential host information or even target its integrity. A malicious container can target the host’s availability by consuming its resources.

4) Use Case IV: Protecting Containers from the Host
Running containers on untrusted hosts should be avoided, especially with the advent of CaaS where containers can be rented from a CaaS provider. In this use case, we assume that containers are honest but the host is either semi-honest or malicious. A semi-honest host can learn about confidential container information because it controls network devices, memory, storage, and processors. A malicious host can also target the integrity of the container and its application(s). Numerous passive and active attacks can be launched from semi-honest and malicious hosts against containers in them. Examples of passive attacks include profiling in-container application activities and unauthorized access for container data. Active attacks can be more harmful since a malicious host can change the application’s behavior.

Software and Hardware Protection Mechanisms

A. Software-Based Protection Mechanisms
Container technology relies heavily on software-based solutions that are either Linux Security Features (LSFs) or Linux Security Modules (LSMs).

1) Linux Kernel Features

a: Namespaces
Namespaces perform the job of isolation and virtualization of system resources for a collection of processes. Namespaces operate as a divider of the identifier tables and other structures linked with kernel global resources into isolated instances. They partition the file systems, processes, users, hostnames, and other components. Hence, each file system namespace will have its private mount table and root directory. For every container, a unique view of the resources can be seen. The constrained view of resources for a process within a container can be extended also to a child process.Namespaces ensure the isolation of processes that are running in a container to blind them from seeing other processes running in a different container.However, one issue with namespaces is that some resources are still not namespace-aware such as devices.There are numerous namespaces available, each of which is responsible for specific resource isolation. 

Solutions that use namespaces
They presented a defense technique that tries to solve the escape attack based on namespace status inspection during process execution.This should help in detecting anomalies and further prevent escape attacks. 

b: Control Groups (CGroups)
CGroups are Linux features that control the accountability and limitation of resource usage such as central processing unit (CPU) runtime, system memory, input/output (I/O), and network bandwidth. In contrast to namespaces, CGroups limit how many resources can be used while namespaces control what resources a container can see (i.e., isolation).Additionally, CGroups prevent containers from using all the available resources and starving other processes.A CGroup is arranged as a slice for each resource. Then a task set can be attached to each specific CGroup. Thus, task groups are forced to use their own part of the resource

Inter-container protection using CGroups (Use case II)
CGroups are vital for inter-container protection requirements. As discussed, CGroups circumscribe the allowed resource usage, hence, a container cannot use more resources than what is designated for it. This helps protect other containers from numerous attacks such as Denial-of-Service (DoS) attacks. For example, a container may use so much of the host’s available RAM that other containers cannot operate properly

Protecting the host from containers using CGroups (Use case III)
CGroups force resource usage limits on containers. This helps prevent the container from performing a DoS attack on the host itself. Additionally, CGroups give the host the power to not only limit resource usage but also account for how much of the resource is used. This could help in implementing usage quotas which can be a very important factor in the emerging cloud computing delivery model CaaS. 

Solutions that use CGroups
Then, for memory DoS protection, the authors demonstrated experimentally that CGroups are not efficient because although they can restrict the allocated memory bandwidth, malicious applications could still use intensive access for that amount of memory. Hence, to protect memory from DoS attacks they used a MemGuard kernel module to prevent each CPU core from exceeding memory access.

c: Capabilities
Linux systems implement the binary option of root and non-root dichotomy. In the context of containers, those binary options can be troublesome. For example, a web server (e.g., Apache) needs to bind on a specific port (e.g., TCP port 80). Without using capabilities, the web server process should have root access to perform its task. This poses a great danger because if it gets compromised, an attacker will be able to control the entire system. Capabilities turn the root and non-root dichotomy into fine-grained access control.Hence, containers (usually not the daemon or container manager) will not need to have full root privilege (assuming there is an available capability for the required tasks). There are thirty-eight capabilities that cover a wide variety of tasks 
For our previous example about web servers, a container can assign the process the CAP_NET_BIND_SERVICE capability. This allows the container to run a version of that web server without requiring full root access. Assuming the web server contains a vulnerability that has been exploited by an adversary, having this capability in place will circumscribe the adversary to a single root operation (i.e., binding ports). On the other hand, if the capability was not set, the compromised or malicious application can perform full root operations on the container.
In addition, capabilities are important for protecting the host from containers.In the previous example, we saw that a container can limit its applications but what happens if the container itself is malicious? This causes a direct risk to the host. Thus, a set of capabilities can be assigned to the container which could reduce the container’s root operational threats.

d: Secure Computation Mode (Seccomp)
Seccomp is a Linux kernel feature that filters system calls to the kernel. Secomp is more fine-grained than capabilities.This helps to decrease the number of system calls coming from containers, which could further reduce possible threats since most attacks leverage kernel exploits through system calls.

Solutions that use Seccomp
Split-Phase Execution of Application Containers (SPEAKER) aiming to differentiate between necessary and unnecessary system calls made by containers.
a solution to mine sandboxes for containers based on automatic testing. During the testing phase, the proposed solution extracts the used system calls by the container. Using Seccomp, the solution creates a profile for each application based on the seen system calls during the testing phase and denies all other calls

e: Namespaces, CGroups, Seccomp, and Capabilities Use in Docker
Docker uses namespaces and CGroups to create a safe virtual environment for its containers.For example, to offer a separate network for each container, the network namespace isolates some network resources like Internet protocol (IP) addresses.Docker depends on CGroups to group processes running in the container. CGroups reveal metrics about CPU and I/O block usage along with managing the resources of Docker such as CPU and memory. Docker resource configurations are either with hard or soft limits. Hard limits are used to specify a specific amount of resources to the container. Soft limits give the container the necessary resources in the machine.The issue with capabilities as expressed by the Docker team in is that the default set of capabilities might not provide complete security isolation.Docker adds support for adding and removing capabilities. Additionally, users can set their own profile.We should note here that as the number of capabilities added to the container increases there will be a higher security risk. This is because the container will be able to perform more root privileged tasks.The default seccomp profile of Docker blocks 44 out of 300 available system calls

2) Linux Security Modules (LSMs)
enhanced access control mechanisms are not accepted widely to maintain OSs. This is due to the fact that there is no consensus on the right solution within the security community.LSMs allow a wide variety of security models to be implemented on Linux kernel as loadable modules.This means that a user can select the preferred implementation rather than being forced to use the one that came with the OS. LSMs focus on providing the needs for implementing Mandatory Access Control (MAC) with minimal changes to the Kernel itself.In case of not specifically selecting an LSM then the default LSM will be the Linux capabilities system.One of the primary issues with LSMs is that they are shared by all containers, wherein a single container cannot use a specific LSM.


B. Hardware-Based Protection Mechanisms
This section addresses solutions to protect containers from a semi-honest or malicious host as well as other containers.We address two available mechanisms: The use of Virtual Trusted Platform Modules (vTPMs) and the use of Intel SGX as a trusted platform support mechanism
1) Virtual Trusted Platform Modules (vTPM)
One commonly used technique in trusted computing is Trusted Platform Modules (TPMs) which is hardware that is intended to be used as a cryptographic processor. TPMs provide hardware support for attestation, sealing data, secure boot, and algorithm acceleration
a: vTPM in Host OS Kernel
b: vTPM in a Dedicated Container
2) Intel SGX
Intel SGX is a set of extensions that allows Intel architecture hardware to provide confidentiality and integrity guarantees when the underlying privileged software (e.g., hypervisor, kernel) is potentially malicious. Intel SGX seamlessly protects containers from underlying layers (e.g., cloud provider, or host machine).

Other Aspects of Containers Security
There are two other important aspects to maintain container security: vulnerabilities management and standard guidelines.

Open Issues and Future Research Directions

A. Meltdown and Spectre Attacks
Meltdown exploits the out-of-order execution in modern processors to extract information about the OS and other containers. Containers are based on the concept of sharing the same kernel.Spectre is another serious threat to containers, as it tricks other applications into accessing arbitrary locations in their memory

B. Standards for Container Security
Further investigation for standardization of container deployment, communication protocols, and assessment methodologies is required

C. Digital Forensics for Containers
Digital forensics is used to analyze security incidents. As the world is moving towards microservices that use containers, digital forensic techniques should be able to analyze security incidents related to them

D. Usability of Vulnerabilities Assessment Tools
Several researchers showed that Docker images contain many high-risk vulnerabilities 

E. Container Alternatives
The study shows that VMs and unikernels provide good isolation compared to containers. However, VMs are inefficient because they are large, unikernels are not optimal because they lack privilege levels (which help separate kernel code from application code), and containers do not provide as good isolation as VMs or unikernels. 

F. Container Security and Privacy for IoT Applications
we re-emphasize the importance of studying container security, privacy, and standardization technologies for different IoT applications such as smart grids, smart vehicles, augmented reality, smart sensor networks, E-Health, and network function virtualization (NFV).

G. Blockchain for Container Verification
Notary can be used to verify Docker images’ authenticity, however, is a centralized solution. A better solution is to use a decentralized verification that could use a blockchain.

H. Container Specific LSMs
One issue with LSMs is that they are shared by all containers, in which a single container cannot use a specific LSM. LSMs play a pivotal role in providing security for Linux systems in general.However, sharing LSMs among different containers can be crippling as different containers might have different protection requirements. 

Conclusions
Containers are important for the future of cloud computing. Microservices and containers are closely related, where containers are considered the standardized way for microservice deployment. Containers are important for the emerging field of service meshes that relies on microservices, too. 

https://ieeexplore.ieee.org/document/8693491

Predictive Analytics


  • Lyric Analysis: Predictive Analytics using Machine Learning with R

Use a variety of machine learning (ML) classification algorithms to build models step-by-step that predict the genre of a song and whether it will be successful on the Billboard charts - based entirely on lyrics
This tutorial explains and provides a musical use case for a form of supervised learning, specifically classification, that is based on the lyrics of a variety of artists (and a couple of book authors).
In this tutorial, you have built a model to predict the genre of a song based entirely on lyrics. You used supervised machine learning classification algorithms and trained models on a set of five different artists and five different genres. By using the mlr framework, you created tasks, learners and resampling strategies to train and then tune a model(s). Then you ran your model against an unseen test dataset of different artists. You were able to identify which algorithms work better with the default settings, and eventually, predict the genre of new songs that your model has never seen
https://www.datacamp.com/community/tutorials/predictive-analytics-machine-learning



  • Machine Learning vs Predictive Analytics – 7 Useful Differences


Predictive analytics is also a part of machine learning domain which is limited to predict future outcome from data based on previous patterns.
While predictive analytics has been in use since more than two decades mainly in banking and finance sector, application of machine learning has taken prominence in recent time with algorithms like object detection from images, text classification, and recommendation systems.

Machine learning internally uses statistics, mathematics, and computer science fundamentals to build logic for algorithms that can do classification, prediction, and optimization in both real times as well as batch mode.

Classification
we tend to classify an object based on its various properties into one or more classes.
There are many standard machine learning algorithms which are used to solve classification problem. Logistic regression is one such method, probably most widely used and most well know, also the oldest. Apart from that we also have some of the most advanced and complicated models ranging from decision tree to random forest, AdaBoost, XP boost, support vector machines, naïve baize and neural network. Since the last couple of years, deep learning is running at the forefront. Typically neural network and deep learning are used to classify images. If there are hundred thousand images of cats and dog and you want to write a code that can automatically separate images of cats and dog, you may want to go for deep learning methods like a convolutional neural network.

Regression
Regression is another class of problem in machine learning where we try to predict a continuous value of a variable instead of a class unlike in classification problem.  Regression techniques are generally used to predict the share price of a stock, sale price of a house or car, a demand for a certain item etc.

Predictive Analytics
There are some areas of overlap between machine learning and predictive analytics. While common techniques like logistic and linear regression come under both machine learning and predictive analytics, advanced algorithms like a decision tree, random forest etc. are essentially machine learning. Under predictive analytics, the goal of the problems remains very narrow where the intent is to compute a value of a particular variable at a future point of time. Predictive analytics is heavily statistics loaded while machine learning is more of a blend of statistics, programming, and mathematics.
https://www.educba.com/machine-learning-vs-predictive-analytics/



  • Big Data

Big Data has emerged over the last years as a concept to handle data that requires new data modeling concepts, data structures, algorithms and/or large-scale distributed clusters.

OLTP has been around for a long time and focuses on transaction processing. When the concept of OLTP emerged it has been usually a synonym for simply using relational databases to store various information related to an application – most people forgot that it was related to processing of transactions. Additionally, it was not about technical database transactions, but business transactions, such as ordering products or receiving money. Nevertheless, most relational databases secure business transactions via technical transactions by adhering to the ACID criteria.
Today OLTP is relevant given its numerous implementations in enterprise systems, such as Enterprise Resource Management systems, Customer Relationship Management systems or Supply Chain Management systems

OLAP has been around nearly as long as OLTP, because most analysis have been done on historized transactional data. Due to the historization and different analysis needs the amount of data is significant higher than in OLTP systems. However, OLAP has a different access pattern: Less concurrent users, but they are interested in the whole set of data, because they want to generate aggregated statistics for them. Hence, a lot of data is usually transferred into an OLAP system from different source systems and afterwards it is only read very often

Going beyond OLTP and OLAP
Aspect 1: Predictive Analytics
Data scientists employing predictive analytics are using statistic and machine learning techniques to predict how a situation may evolve in the future.
Some of these techniques exist already since decades, but only since recently they make more sense, because more data can be processed with Big Data technologies.
However, current Big Data technologies, such as Hadoop, are not transparent to the end user. This is not really an issue with the Big Data technologies themselves, but with the tools used for accessing and processing the data, such as R, Matlab or SAS.

https://jornfranke.wordpress.com/2015/06/28/big-data-what-is-next-oltp-olap-predictive-analytics-sampling-and-probabilistic-databases/

  • Advanced Analytics

Advanced Analytics is the autonomous or semi-autonomous examination of data or content using sophisticated techniques and tools, typically beyond those of traditional business intelligence (BI), to discover deeper insights, make predictions, or generate recommendations. Advanced analytic techniques include those such as data/text mining, machine learning, pattern matching, forecasting, visualization, semantic analysis, sentiment analysis, network and cluster analysis, multivariate statistics, graph analysis, simulation, complex event processing, neural networks.
https://www.gartner.com/it-glossary/advanced-analytics/



  • What is advanced analytics compared to business intelligence?


Business Intelligence – traditionally focuses on using a consistent set of metrics to measure past performance and guide business planning. Business Intelligence consists of querying, reporting, OLAP (online analytical processing), and can answer questions including “what happened,” “how many,” and “how often.”

Advanced Analytics – goes beyond Business Intelligence by using sophisticated modeling techniques to predict future events or discover patterns which cannot be detected otherwise. Advanced Analytics can answer questions including “why is this happening,” “what if these trends continue,” “what will happen next” (prediction), “what is the best that can happen” (optimization).

Advanced Analytics vs BI
Where Business Intelligence is focused on reporting and querying, Advanced Analytics is about optimizing, correlating, and predicting the next best action or the next most likely action
https://rapidminer.com/glossary/advanced-analytics-vs-bi/


  • What is Advanced and Predictive Analytics?

Advanced analytics describes data analysis that goes beyond simple mathematical calculations such as sums and averages, or filtering and sorting. Advanced analyses use mathematical and statistical formulas and algorithms to generate new information, to recognize patterns, and also to predict outcomes and their respective probabilities.
Predictive analytics is a sub-division of advanced analytics and focuses on the identification of future events and values with their respective probabilities.
https://bi-survey.com/predictive-analytics

Sunday, May 26, 2019

Statement of Work (SOW)


  • Definition

statement of work (SOW)
A statement of work (SOW), in project management, is a document in which a contracting officer or chief procurement officer (CPO) specifies the objectives and deliverables for a particular project or service contract. An SOW is often included as part of a request for proposal (RFP), a document used to solicit sealed bids from potential vendors and service providers.


What is included in a statement of work document

Background- This section of a statement of work explains the context for the project and documents the project's overarching goals and requirements
Purpose/objectives- This sections states the project’s overarching goals and how they will solve business problems or positively affect different parts of the organization.
Scope of work- This section of an SOW will document what work will be performed under a contractual agreement, how the work will be divided and who is responsible for completing the work
Tasking and deliverables- This section defines the specific tasks or deliverables the contractor must perform, along with a timeline for work to be completed.
Standards and testing- This section outlines any industry or compliance standards that must be met when executing the project as well as any testing that needs to be done.
Acceptance criteria- This section specifies how the customer will determine whether or not the contractor or service provider has met the objectives of project tasks and deliverables.
Payment- This section documents how and when completed work will be invoiced and when payment will be scheduled.

There are three common types of SOW documents.
Design/detail- This model of SOW focuses on the details behind the project requirements and processes
Level of effort/time and materials/unit rate- This is the most common version of the SOW that is typically used as a template for most projects
Performance- This type of SOW is performance-based and focuses on the purpose and ends results of the project.

https://searchitchannel.techtarget.com/definition/statement-of-work-SOW


  • What is a Statement of Work?

A Statement of Work (SOW) is a document within a contract that describes the work requirements for a specific project along with its performance and design expectations.
The main purpose of the SOW is to define the liabilities, responsibilities and work agreements between two parties, usually clients and service providers.

A well-written SOW will define the scope and Key Performance Indicators (KPIs) for the agreement.
These KPIs can then be used as a baseline to determine whether the service provider has met conditions of the SOW.

https://www.villanovau.com/resources/contract-management/what-is-statement-of-work/



  • What Is a Statement of Work?

The SoW is the document that captures and defines all aspects of your project. You’ll note the activities, deliverables and the timetable for the project.

What Is the Use of a Statement of Work?
As noted, the statement of work is a detailed overview of the project in all its dimensions. It’s also a way to share what the project entails with those who are working on the project, whether they are collaborating or are contracted to work on the project. This includes vendors and contractors who are bidding to work on the project

It’s also helpful to the project leader, as it provides a structure on which the project plan can be built on. The statement of work will also help to avoid conflicts in the project. With detail and clarity, the SoW helps keep everyone that’s involved in the project on the same page and works to leave confusion to a minimum.

Different Examples of a Statement of Work
There are three main types

Design/Detail: When you’re writing this SoW what you’re doing is conveying to the supplier how you want the work done. What are the buyer requirements that will control the supplier’s process?

Level of Effort/Time and Materials/Unit Rate: This is an almost universal version and it can apply to most projects. What it defines is hourly service as well as those materials required to perform the tasks. It tends to find use in short-term contracts.

Performance-Based: This is the preferred SoW of project managers as it focuses on the purpose of the project, the resources and the quality level expected of the deliverables. It does not, however, explain how someone supposes the work to get done. This allows a great deal of autonomy on how to get to an outcome without requiring a specific process.

https://www.projectmanager.com/blog/statement-work-definition-examples


Thursday, May 23, 2019

Code Coverage


  • In computer science, test coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs

A program with high test coverage, measured as a percentage, has had more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low test coverage
Basic coverage criteria
There are a number of coverage criteria, the main ones being:
    Function coverage – Has each function (or subroutine) in the program been called?
    Statement coverage – Has each statement in the program been executed?
    Edge coverage – has every edge in the Control flow graph been executed?
    Branch coverage – Has each branch (also called DD-path) of each control structure (such as in if and case statements) been executed? For example, given an if statement, have both the true and false branches been executed? Notice that this one is a subset of Edge coverage.
    Condition coverage (or predicate coverage) – Has each Boolean sub-expression evaluated both to true and false?
https://en.wikipedia.org/wiki/Code_coverage
  • What is Code Coverage?

Code coverage is the percentage of code which is covered by automated tests. Code coverage measurement simply determines which statements in a body of code have been executed through a test run, and which statements have not. In general, a code coverage system collects information about the running program and then combines that with source information to generate a report on the test suite's code coverage.
Code coverage is part of a feedback loop in the development process. As tests are developed, code coverage highlights aspects of the code which may not be adequately tested and which require additional testing. This loop will continue until coverage meets some specified target.

Why Measure Code Coverage?
It is well understood that unit testing improves the quality and predictability of your software releases
Measuring code coverage can keep your testing up to the standards you require.

Do you know, however, how well your unit tests actually test your code?
How many tests are enough?
Do you need more tests?

In summary, we measure code coverage for the following reasons:
    To know how well our tests actually test our code
    To know whether we have enough testing in place
    To maintain the test quality over the lifecycle of a project


https://confluence.atlassian.com/clover/about-code-coverage-71599496.html

  • Use code coverage to determine how much code is being tested

To determine what proportion of your project's code is actually being tested by coded tests such as unit tests, you can use the code coverage feature of Visual Studio. To guard effectively against bugs, your tests should exercise or 'cover' a large proportion of your code
https://docs.microsoft.com/en-us/visualstudio/test/using-code-coverage-to-determine-how-much-code-is-being-tested?view=vs-2017

  • Code Coverage vs. Test Coverage: Pros and Cons


Code Coverage
This metric aims to measure number of lines covered by the test cases. It reports total number of lines in the code and number of lines executed by tests. Think of it as the degree to which the source code of a program is executed when a test suite runs. The intent is, the higher the code coverage, the lower the chance of having undetected software bugs.
Subtypes

Coverage is further split into many subtypes – function coverage, branch coverage, condition coverage, loop coverage, statement coverage and parameter value coverage.

Let me define the subtypes for the sake of clarity.

    If tests cover all the function calls in code, function coverage is said to be 100%.
    Similarly, if all branches in the code, i.e. all the if – else conditions have been tested with every possible input, then branch coverage is said to be 100%.
    If all loop statements in the code are executed then loop coverage and statement coverage would be 100% respectively

For Python, a popular tool for code coverage measurement is Coverage.py
A measurement tool widely used for Go is gocov.


Test Coverage

Test Coverage aims to measure of the effectiveness of testing in a qualitative manner. It determines whether the test cases are covering entire functional requirements. You can think of it as a kind of black box testing, where test cases are not written based on code but based on user requirements or expected functionality.
Subtypes
The common mechanisms used for test coverage measurement are unit testing, functional testing, performance testing, integration or system testing and acceptance testing.

Unit tests are written at a granular level to check if a function/method performs as expected.
In functional testing, each functionality mentioned in the requirement document is tested.
Performance testing is generally a way to stress test the code for its responsiveness and stability at different workloads.
Integration testing or system testing is done to test if completely integrated product works in expected manner.
Acceptance testing is generally done at the end of the development cycle. For acceptance testing, the product is handed over to the stakeholders and tested by them to determine whether the product is in acceptable state.

For Python, framework called PyUnit is available.
JUnit, the well known Java testing framework serves as a foundation to launch tests on the Java Virtual Machine.
Go also offers a built in testing package for programs letting you easily write, organize and run tests


Code Coverage vs Test Coverage
So, now we know that code coverage is a measure of how much code is executed during testing, while test coverage is a measure of how much of the feature set is covered with tests.

Code Coverage Pros and Cons
Pros:
Helps capture bugs in program flow.
Cons:
Most Code coverage tools are relevant for Unit test only. It is essential to track all test types coverage (unit, API, System and even manual).

Test Coverage Pros and Cons
As a black-box testing methodology this approach is aligned to the requirement specifications and has minimal interaction with code itself.
Cons:
Most of the tasks in such testing methodology are manual.

Function and feature are different. Covering a single feature will cover several functions, while the required effort may be similar.

https://www.sealights.io/test-metrics/code-coverage-vs-test-coverage-pros-and-cons
  • Code coverage is a metric that can help you understand how much of your source is tested

Code coverage tools will use one or more criteria to determine how your code was exercised or not during the execution of your test suite.

Function coverage: how many of the functions defined have been called.
Statement coverage: how many of the statements in the program have been executed.
Branches coverage: how many of the branches of the control structures (if statements for instance) have been executed.
Condition coverage: how many of the boolean sub-expressions have been tested for a true and a false value.
Line coverage: how many of lines of source code have been tested

Code coverage tools can help you understand where you should focus your attention next, but they won't tell you if your existing tests are robust enough for unexpected behaviors.
Achieving great coverage is an excellent goal, but it should be paired with having a robust test suite that can ensure that individual classes are not broken as well as verify the integrity of the system
When you've established your continuous integration (CI) workflow you can start failing the tests if you don't reach a high enough percentage of coverage. Of course, as we said it earlier, it would be unreasonable to set the failure threshold too high, and 90% coverage is likely to cause your build to fail a lot. If your goal is 80% coverage, you might consider setting a failure threshold at 70% as a safety net for your CI culture.
Once again, be careful to avoid sending the wrong message as pressuring your team to reach good coverage might lead to bad testing practices
https://www.atlassian.com/continuous-delivery/software-testing/code-coverage
  • Linting is the process of checking the source code for Programmatic as well as Stylistic errors. This is most helpful in identifying some common and uncommon mistakes that are made during coding
  • A Lint or a Linter is a program that supports linting (verifying code quality). They are available for most languages like JavaScript, CSS, HTML, Python, etc..
    https://stackoverflow.com/questions/8503559/what-is-linting/8503586

  • A linter or lint refers to tools that analyze source code to flag programming errors, bugs, stylistic errors, and suspicious constructs.The term is originated from a Unix utility that examined C language source code.

  • https://en.wikipedia.org/wiki/Lint_(software)

  • EclEmma
EclEmma is an Eclipse plug-in that generates code coverage reports and provide simple trace information about test cases
Coverage is defined as a measure of the completeness of the set of test cases.
This definition means that the more source code that is executed by your test cases the better.
There are four types of coverage:
    Method coverage: how many of your methods have been called by your test cases?
    Statement coverage: how many of your statements have been run by your test cases?
    Decision/Branch coverage: how many decision points (if statements) have evaluated both true and false?
    Condition coverage: how many Boolean sub-expressions have been evaluated both true and false at a decision point?
EclEmma may be used on top of JUnit.
EclEmma runs the JUnit test cases and generates the coverage report from the execution.
EclEmma provides overall, package, and file level information on the statement.
http://agile.csc.ncsu.edu/SEMaterials/tutorials/eclemma/


  • Coverage.py

Coverage measurement is typically used to gauge the effectiveness of tests. It can show which parts of your code are being exercised by tests, and which are not.
https://coverage.readthedocs.io/en/v4.5.x/
  • gocov

Coverage reporting tool for The Go Programming Language
https://github.com/axw/gocov
  • EMMA

EMMA is a fast Java code coverage tool based on bytecode instrumentation. It differs from the existing tools by enabling coverage profiling on large scale enterprise software projects with simultaneous emphasis on fast individual development.
http://sourceforge.net/projects/emma/


  • Crap4j 

Crap4j report screenshot. Crap4j is a Java implementation of the CRAP (Change Risk Analysis and Predictions) software metric. The CRAP metric combines cyclomatic complexity and code coverage from automated tests (e.g. JUnit tests) to help you identify code that might be particularly difficult to understand, test, or maintain
www.crap4j.org
  • Cobertura
Cobertura is a Java code coverage reporting tool. It is based on jcoverage and works on any platform with Java 5 or higher. You can use Cobertura with ant, maven or command line access.
Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage.
http://cobertura.github.io/cobertura/


  • djUnit

djUnit uses jcoverage to generate a coverage report of the performed tests, and it can be checked on Eclipse.
The results are shown on djUnit coverage view , task list and java source editor.

http://works.dgic.co.jp/djunit/
  • Lint4j
Lint4j ("Lint for Java") is a static Java source and byte code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis.

http://www.jutils.com/

  • JaCoCo - Java Code Coverage Library

https://www.jacoco.org/jacoco/trunk/index.html