GitOps is a way of implementing Continuous Deployment for cloud native applications. It focuses on a developer-centric experience when operating infrastructure, by using tools developers are already familiar with, including Git and Continuous Deployment tools.
The core idea of GitOps is having a Git repository that always contains declarative descriptions of the infrastructure currently desired in the production environment and an automated process to make the production environment match the described state in the repository. If you want to deploy a new application or update an existing one, you only need to update the repository - the automated process handles everything else. It’s like having cruise control for managing your applications in production.
Why should I use GitOps?
Deploy Faster More Often
What is unique about GitOps is that you don’t have to switch tools for deploying your application. Everything happens in the version control system you use for developing the application anyways.
When we say “high velocity” we mean that every product team can safely ship updates many times a day — deploy instantly, observe the results in real time, and use this feedback to roll forward or back.
Easy and Fast Error Recovery
This makes error recovery as easy as issuing a git revert and watching your environment being restored.The Git record is then not just an audit log but also a transaction log. You can roll back & forth to any snapshot.
Easier Credential Management
your environment only needs access to your repository and image registry. That’s it. You don’t have to give your developers direct access to the environment.
kubectl is the new ssh. Limit access and only use it for deployments when better tooling is not available.
Self-documenting Deployments
With GitOps, every change to any environment must happen through the repository. You can always check out the master branch and get a complete description of what is deployed where plus the complete history of every change ever made to the system. And you get an audit trail of any changes in your system
Shared Knowledge in Teams
Using Git to store complete descriptions of your deployed infrastructure allows everybody in your team to check out its evolution over time. With great commit messages everybody can reproduce the thought process of changing infrastructure and also easily find examples of how to set up new systems.
How does GitOps work?
Environment Configurations as Git repository
GitOps organizes the deployment process around code repositories as the central element. There are at least two repositories: the application repository and the environment configuration repository. The application repository contains the source code of the application and the deployment manifests to deploy the application. The environment configuration repository contains all deployment manifests of the currently desired infrastructure of an deployment environment.
Push-based vs. Pull-based Deployments
There are two ways to implement the deployment strategy for GitOps: Push-based and Pull-based deployments.
The difference between the two deployment types is how it is ensured, that the deployment environment actually resembles the desired infrastructure.
When possible, the Pull-based approach should be preferred as it is considered the more secure and thus better practice to implement GitOps.
Push-based Deployments
The Push-based deployment strategy is implemented by popular CI/CD tools such as Jenkins, CircleCI, or Travis CI. The source code of the application lives inside the application repository along with the Kubernetes YAMLs needed to deploy the app. Whenever the application code is updated, the build pipeline is triggered, which builds the container images and finally the environment configuration repository is updated with new deployment descriptors.
With this approach it is indispensable to provide credentials to the deployment environment.
In some use cases a Push-based deployment is inevitable when running an automated provisioning of cloud infrastructure. In such cases it is strongly recommended to utilize the fine-granular configurable authorization system of the cloud provider for more restrictive deployment permissions.
Another important thing to keep in mind when using this approach is that the deployment pipeline only is triggered when the environment repository changes. It can not automatically notice any deviations of the environment and its desired state. This means, it needs some way of monitoring in place, so that one can intervene if the environment doesn’t match what is described in the environment repository
Pull-based Deployments
The Pull-based deployment strategy uses the same concepts as the push-based variant but differs in how the deployment pipeline works. Traditional CI/CD pipelines are triggered by an external event, for example when new code is pushed to an application repository. With the pull-based deployment approach, the operator is introduced. It takes over the role of the pipeline by continuously comparing the desired state in the environment repository with the actual state in the deployed infrastructure. Whenever differences are noticed, the operator updates the infrastructure to match the environment repository. Additionally the image registry can be monitored to find new versions of images to deploy.
Just like the push-based deployment, this variant updates the environment whenever the environment repository changes. However, with the operator, changes can also be noticed in the other direction. Whenever the deployed infrastructure changes in any way not described in the environment repository, these changes are reverted. This ensures that all changes are made traceable in the Git log, by making all direct changes to the cluster impossible.
This change in direction solves the problem of push-based deployments, where the environment is only updated when the environment repository is updated.
Most operators support sending mails or Slack notifications if it can not bring the environment to the desired state for any reason, for example if it can not pull a container image.
Additionally, you probably should set up monitoring for the operator itself, as there is no longer any automated deployment process without it.
The operator should always live in the same environment or cluster as the application to deploy
seen with the push-based approach, where credentials for doing deployments are known by the CI/CD pipeline.
When the actual deploying instance lives inside the very same environment, no credentials need to be known by external services.
The Authorization mechanism of the deployment platform in use can be utilized to restrict the permissions on performing deployments.
When using Kubernetes, RBAC configurations and service accounts can be utilized.
Working with Multiple Applications and Environments
When you are using a microservices architecture, you probably want to keep each service in its own repository.
GitOps can also handle such a use case. You can always just set up multiple build pipelines that update the environment repository. From there on the regular automated GitOps workflow kicks in and deploys all parts of your application.
Managing multiple environments with GitOps can be done by just using separate branches in the environment repository.
You can set up the operator or the deployment pipeline to react to changes on one branch by deploying to the production environment and another to deploy to staging.
FAQ
Is my project ready for GitOps?
All you need to get started is infrastructure that can be managed with declarative Infrastructure as Code tools.
I don’t use Kubernetes. Can I still use GitOps?
In principle, you can use any infrastructure that can be observed and described declaratively, and has Infrastructure as Code tools available
However, currently most operators for pull-based GitOps are implemented with Kubernetes in mind.
Is GitOps just versioned Infrastructure as Code?
Declarative Infrastructure as Code plays a huge role for implementing GitOps,
GitOps takes the whole ecosystem and tooling around Git and applies it to infrastructure.
Continuous Deployment systems guarantee that the currently desired state of the infrastructure is deployed in the production environment.
Apart from that you gain all the benefits of code reviews, pull requests, and comments on changes for your infrastructure.
How to get secrets into the environment without storing them in git?
you have have secrets created within the environment which never leave the environment
For example, you provision a database within the environment and give the secret to the applications interacting with the database only.
Another approach is to add a private key once to the environment (probably by someone from a dedicated ops team) and from that point you can add secrets encrypted by the public key to the environment repository.
How does GitOps Handle DEV to PROD Propagation?
GitOps doesn’t provide a solution to propagating changes from one stage to the next one. We recommend using only a single environment and avoid stage propagation altogether. But if you need multiple stages (e.g., DEV, QA, PROD, etc.) with an environment for each, you need to handle the propagation outside of the GitOps scope, for example by some CI/CD pipeline.
We are already doing DevOps. What’s the difference to GitOps?
GitOps is a technique to implement Continuous Delivery. While DevOps and GitOps share principles like automation and self-serviced infrastructure, these shared principles certainly make it easier to adopt a GitOps workflow when you are already actively employing DevOps techniques.
So, is GitOps basically NoOps?
You can use GitOps to implement NoOps,
If you are using cloud resources anyway, GitOps can be used to automate those.
Typically, however, some part of the infrastructure like the network configuration or the Kubernetes cluster you use isn’t managed by yourself decentrally but rather managed centrally by some operations team.
So operations never really goes away.
Is there also SVNOps?
If you prefer SVN over Git,you may need to put more effort into finding tools that work for you or even write your own.
All available operators only work with Git repository
https://www.gitops.tech/
ArgoCD: A GitOps operator for Kubernetes with a web interface
Flux: The GitOps Kubernetes operator by the creators of GitOps — Weaveworks
Gitkube: A tool for building and deploying docker images on Kubernetes using git push
JenkinsX: Continuous Delivery on Kubernetes with built-in GitOps
Terragrunt: A wrapper for Terraform for keeping configurations DRY, and managing remote state
WKSctl: A tool for Kubernetes cluster configuration management based on GitOps principles
Helm Operator: An operator for using GitOps on K8s with Helm
werf: A CLI tool to build images and deploy them to Kubernetes via push-based approach
A container runtime a lower level component typically used in a Container Engine but can also be used by hand for testing. The Open Containers Initiative (OCI) Runtime Standard reference implementation is runc. This is the most widely used container runtime, but there are others OCI compliant runtimes, such as crun, railcar, and katacontainers. Docker, CRI-O, and many other Container Engines rely on runc.
Kernel Namespace
When discussing containers, Kernel namespaces are perhaps the most important data structure, because they enable containers as we know them today. Kernel namespaces enable each container to have it’s own mount points, network interfaces, user identifiers, process identifiers, etc.
When you type a command in a Bash terminal and hit enter, Bash makes a request to the kernel to create a normal Linux process using a version of the exec() system call. A container is special because when you send a request to a container engine like docker, the docker daemon makes a request to the kernel to create a containerized process using a different system call called clone(). This clone() system call is special because it can create a process with its own virtual mount points, process ids, user ids, network interfaces, hostname, etc
In computer programming, a runtime system, also called runtime environment, primarily implements portions of an execution model.Most programming languages have some form of runtime system that provides an environment in which programs run. This environment may address a number of issues including the management of application memory, how the program accesses variables, mechanisms for passing parameters between procedures, interfacing with the operating system, and otherwise. The compiler makes assumptions depending on the specific runtime system to generate correct code. Typically the runtime system will have some responsibility for setting up and managing the stack and heap, and may include features such as garbage collection, threads or other dynamic features built into the language
https://en.wikipedia.org/wiki/Runtime_system
User namespaces allow non-root users to pretend to be the root•Root-in-UserNS can have “fake” UID 0 and also create other namespaces (MountNS, NetNS..)
Skopeo for sharing/finding container images on Docker registries, the Atomic registry, private registries, local directories and local OCI-layout directories.
Podman for running containers without need for daemon.
https://computingforgeeks.com/how-to-install-podman-on-ubuntu/
How Docker CLI Works
The Docker CLI is a client/server operation and the Docker CLI communicates with the Docker engine when it wants to create or manipulate the operations of a container. This client/server architecture can lead into problems in production because one, you have to start the Docker daemon before Docker CLI comes alive. The Docker CLI then sends an API call to the Docker Engine to launch Open Container Initiative (OCI) Container runtime, in most cases runc, to start the container (projectatomic.io). What this means is that the launched containers are child processes of the Docker Engine.
What is Podman?
What then is Podman? Podman is a daemonless container engine for developing, managing, and running OCI Containers on your Linux System
Docker vs Podman
The major difference between Docker and Podman is that there is no daemon in Podman. It uses container runtimes as well for example runc but the launched containers are direct descendants of the podman process. This kind of architecture has its advantages such as the following:
Applied Cgroups or security constraints still control the container: Whatever cgroup constraints you apply on the podman command, the containers launched will receive those same constraints directly.
Advanced features of systemd can be utilized using this model: This can be done by placing podman into a systemd unit file and hence achieving more.
What about Libpod?
Libpod just provides a library for applications looking to use the Container Pod concept, popularized by Kubernetes.
It allows other tools to manage pods/container (projectatomic.io). Podman is the default CLI tool for using this library.
There are other two important Libraries that make Podman possible:
containers/storage – This library allows one to use copy-on-write (COW) file systems, required to run containers.
containers/image – This library that allows one to download and install OCI Based Container Images from containers registries like Docker.io, Quay, and Artifactory, as well as many others (projectatomic.io).
A good example is that you can be running a full Kubernetes environment with CRI-O, building container images using Buildah and managing your containers and pods with Podman at the same time (projectatomic.io).
Podman provides some extra features that help developers and operators in Kubernetes environments. There are extra commands provided by Podman that are not available in Docker. If you
are familiar with Docker and are considering using Kubernetes/OpenShift as your container platform, then Podman can help you.
Podman can generate a Kubernetes YAML file based on a running container using podman generate kube. The command podman pod can be used to help debug running Kubernetes pods along with the standard container commands.
You can either build using a Dockerfile using podman build or you can run a container and make lots of changes and then commit those changes to a new image tag
What is Buildah and why would I use it?
Podman does do builds and for those familiar with Docker, the build process is the same. You can either build using a Dockerfile using podman build or you can run a container and make lots of changes and then commit those changes to a new image tag
Buildah can be described as a superset of commands related to creating and managing container images and, therefore, it has much finer-grained control over images. Podman’s build command contains a subset of the Buildah functionality. It uses the same code as Buildah for building.
The most powerful way to use Buildah is to write Bash scripts for creating your images—in a similar way that you would write a Dockerfile.
When Kubernetes moved to CRI-O based on the OCI runtime specification, there was no need to run a Docker daemon and, therefore, no need to install Docker on any host in the Kubernetes cluster for running pods and containers.
Kubernetes could call CRI-O and it could call runC directly.This, in turn, starts the container processes.
However, if we want to use the same Kubernetes cluster to do builds, as in the case of OpenShift clusters, then we needed a new tool to perform builds that would not require the Docker daemon and subsequently require that Docker be installed.
Such a tool, based on the containers/storage and containers/image projects, would also eliminate the security risk of the open Docker daemon socket during builds, which concerned many users
There are a couple of extra things practitioners need to understand about Buildah:
It allows for finer control of creating image layers.
Buildah’s run command is not the same as Podman’s run command. Because Buildah is for building images, the run command is essentially the same as the Dockerfile RUN command.
Buildah can build images from scratch, that is, images with nothing in them at all.
In fact, looking at the container storage created as a result of a buildah from scratch command yields an empty directory. This is useful for creating very lightweight images that contain only the packages needed in order to run your application.
A good example use case for a scratch build is to consider the development images versus staging or production images of a Java application. During development, a Java application container image may require the Java compiler and Maven and other tools. But in production, you may only require the Java runtime and your packages. And, by the way, you also do not require a package manager such as DNF/YUM or even Bash. Buildah is a powerful CLI for this use case
Now that we had solved the Kubernetes runtime issue with CRI-O and runC, and we had solved the build problem with Buildah, there was still one reason why Docker was still needed on a Kubernetes host: debugging
How can we debug container issues on a host if we don’t have the tools to do it? We would need to install Docker, and then we are back where we started with the Docker daemon on the host. Podman solves this problem.
Podman becomes a tool that solves two problems. It allows operators to examine containers and images with commands they are familiar with using. And it also provides developers with the same tools.
https://developers.redhat.com/blog/2019/02/21/podman-and-buildah-for-docker-users/
When considering how to implement something like this, we considered the following developer and user workflow:
Create containers/pods locally using Podman on the command line.
Verify these containers/pods locally or in a localized container runtime (on a different physical machine).
Snapshot the container and pod descriptions using Podman and help users re-create them in Kubernetes.
Users add sophistication and orchestration (where Podman cannot) to the snapshot descriptions and leverage advanced functions of Kubernetes
https://developers.redhat.com/blog/2019/01/29/podman-kubernetes-yaml/
The main motivation was to move away from the need of having a daemon that requires root access.
Podman, Skopeo and Buildah are a set of tools that you can use to manage and run container images
a service mesh is a dedicated infrastructure layer for handling service-to-service communication. Although this definition sounds very much like a CNI implementation on Kubernetes, there are some differences. A service mesh typically sits on top of the CNI and builds on its capabilities. It also adds several additional capabilities like service discovery and security.
The components of a service mesh include:
Data plane - made up of lightweight proxies that are distributed as sidecars. Proxies include NGINX, or envoy; all of these technologies can be used to build your own service mesh in Kubernetes. In Kubernetes, the proxies are run as cycles and are in every Pod next to your application.
Control plane - provides the configuration for the proxies, issues the TLS certificates authority, and contain the policy managers. It can collect telemetry and other metrics and some service mesh implementations also include the ability to perform tracing.
How is a service mesh useful?
The example shown below illustrates a Kubernetes cluster with an app composed of these services: a front-end, a backend and a database.
What a service mesh provides?
Not all of the services meshes out there have all of these capabilities, but in general, these are the features you gain:
Service Discovery (eventually consistent, distributed cache)
Load Balancing (least request, consistent hashing, zone/latency aware)
Communication Resiliency (retries, timeouts, circuit-breaking, rate limiting)
Security (end-to-end encryption, authorization policies)
Observability (Layer 7 metrics, tracing, alerting)
Routing Control (traffic shifting and mirroring)
API (programmable interface, Kubernetes Custom Resource Definitions (CRD))
Differences between service mesh implementations?
Istio
Has a Go control plane and uses Envoy as a proxy data plane. Istio is a complex system that does many things, like tracing, logging, TLS, authentication, etc. A drawback is the resource hungry control plane
The more services you have the more resources you need to run them on Istio.
AWS App Mesh
still lacks many of the features that Istio has. For example it doesn’t include mTLS or traffic policies.
Linkerd v2
Also has a Go control plane and a Linkerd proxy data plane that is written in Rust.
Linkerd has some distributed tracing capabilities and just recently implemented traffic shifting.
The current 2.4 release implements the Service Mesh Interface (SMI) traffic split API, that makes it possible to automate Canary deployments and other progressive delivery strategies with Linkerd and Flagger.
Consul Connect
Uses a Consul control plane and requires the data plane to managed inside an app. It does not implement Layer 7 traffic management nor does it support Kubernetes CRDs.
How does progressive delivery work with a service mesh?
Progressive delivery is Continuous Delivery with fine-grained control over the blast radius. This means that you can deliver new features of your app to a certain percentage of your user base.
In order to control the progressive deployments, you need the following:
User segmentation (provided by the service mesh)
Traffic shifting Management (provided by the service mesh)
Observability and metrics (provided by the service mesh)
Automation (service mesh add-on like Flagger)
Canary
A canary is used for when you want to test some new functionality typically on the backend of your application. Traditionally you may have had two almost identical servers: one that goes to all users and another with the new features that gets rolled out to a subset of users and then compared. When no errors are reported, the new version can gradually roll out to the rest of the infrastructure.
In the basic architectural diagram above,
the green boxes in the data plane represent applications,
the blue squares are service mesh proxies,
and the rectangles are application endpoints (a pod, a physical host, etc).
The control plane provides a centralized API for controlling proxy behavior in aggregate.
While interactions with the control plane can be automated (e.g. by a CI/CD pipeline), it’s typically where you–as a human–would interact with the service mesh.
Any service mesh in this guide has certain features
Resiliency features (retries, timeouts, deadlines, etc)
Cascading failure prevention (circuit breaking)
Robust load balancing algorithms
Control over request routing (useful for things like CI/CD release patterns)
The ability to introduce and manage TLS termination between communication endpoints
Rich sets of metrics to provide instrumentation at the service-to-service layer
What’s Different About a Service Mesh?
The service mesh exists to make your distributed applications behave reliably in production.
With microservices, service-to-service communication becomes the fundamental determining factor for how your applications behave at runtime.
Application functions that used to occur locally as part of the same runtime instead occur as remote procedure calls being transported over an unreliable network.
Product Comparisons
Linkerd
Built on Twitter’s Finagle library, Linkerd is written in Scala and runs on the JVM
Linkerd includes both a proxying data plane and the Namerd (“namer-dee”) control plane all in one package.
Notable features include:
All of the “table stakes” features (listed above),
Support for multiple platforms (Docker, Kubernetes, DC/OS, Amazon ECS, or any stand-alone machine),
Built-in service discovery abstractions to unite multiple systems,
Support for gRPC, HTTP/2, and HTTP/1.x requests + all TCP traffic.
Envoy
It is written as a high performance C++ application proxy designed for modern cloud-native services architectures.
Envoy is designed to be used either as a standalone proxying layer or as a “universal data plane” for service mesh architectures
Specifically on serving as a foundation for more advanced application proxies, Envoy fills the “data plane” portion of a service mesh architecture.
Envoy is a performant solution with a small resource footprint that makes it amenable to running it as either a shared-proxy or sidecar-proxy deployment mode
You can also find Envoy embedded in security frameworks, gateways, or other service mesh solutions like Istio
Notable features include:
All of the “table stakes” features (when paired with a control plane, like Istio),
Low p99 tail latencies at scale when running under load,
Acts as a L3/L4 filter at its core with many L7 filters provided out of the box,
Support for gRPC, and HTTP/2 (upstream/downstream),
API-driven, dynamic configuration, hot reloads,
Strong focus on metric collection, tracing, and overall observability.
Istio
Istio is designed to provide a universal control plane to manage a variety of underlying service proxies (it pairs with Envoy by default)
Istio initially targeted Kubernetes deployments, but was written from the ground up to be platform agnostic
The Istio control plane is meant to be extensible and is written in Go.
Its design goals mean that components are written for a number of different applications, which is part of what makes it possible to pair Istio with a different underlying data plane, like the commercially-licensed Nginx proxy. Istio must be paired with an underlying proxy.
Notable features include:
All of the table stakes features (when paired with a data plane, like Envoy),
Security features including identity, key management, and RBAC,
Fault injection,
Support for gRPC, HTTP/2, HTTP/1.x, WebSockets, and all TCP traffic,
Sophisticated policy, quota, and rate limiting,
Multi-platform, hybrid deployment.
Conduit
Conduit aims to drastically simplify the service mesh user experience for Kubernetes.
Conduit contains both a data plane (written in Rust) and a control plane (written in Go).
All of the table stakes features (some are pending roadmap items as of Apr 2018),
Extremely fast and predictable performance (sub-1ms p99 latency),
A native Kubernetes user experience (only supports Kubernetes),
Support for gRPC, HTTP/2, and HTTP/1.x requests + all TCP traffic.
https://thenewstack.io/which-service-mesh-should-i-use/
Kubernetes Service Mesh: A Comparison of Istio, Linkerd and Consul
Cloud-native applications are often architected as a constellation of distributed microservices, which are running in Containers.
This exponential growth in microservices creates challenges around figuring out how to enforce and standardize things like routing between multiple services/versions, authentication and authorization, encryption, and load balancing within a Kubernetes cluster.
Building on Service Mesh helps resolve some of these issues, and more. As containers abstract away the operating system from the application, Service Meshes abstract away how inter-process communications are handled.
What is Service Mesh
The thing that is most crucial to understand about microservices is that they are heavily reliant on the network.
Service Mesh manages the network traffic between services.
It does that in a much more graceful and scalable way compared to what would otherwise require a lot of manual, error-prone work and operational burden that is not sustainable in the long-run.
service mesh layers on top of your Kubernetes infrastructure and is making communications between services over the network safe and reliable.
Service mesh allows you to separate the business logic of the application from observability, and network and security policies. It allows you to connect, secure, and monitor your microservices.
Connect: Service Mesh enables services to discover and talk to each other. It enables intelligent routing to control the flow of traffic and API calls between services/endpoints. These also enable advanced deployment strategies such as blue/green, canaries or rolling upgrades, and more.
Secure: Service Mesh allows you secure communication between services. It can enforce policies to allow or deny communication. E.g. you can configure a policy to deny access to production services from a client service running in development environment.
Monitor: Service Mesh enables observability of your distributed microservices system. Service Mesh often integrates out-of-the-box with monitoring and tracing tools (such as Prometheus and Jaeger in the case of Kubernetes) to allow you to discover and visualize dependencies between services, traffic flow, API latencies, and tracing.
Service Mesh Options for Kubernetes:
Consul
Consul is part of HashiCorp’s suite of infrastructure management products
it started as a way to manage services running on Nomad and has grown to support multiple other data center and container management platforms including Kubernetes.
Consul Connect uses an agent installed on every node as a DaemonSet which communicates with the Envoy sidecar proxies that handles routing & forwarding of traffic.
Istio
Istio has separated its data and control planes by using a sidecar loaded proxy which caches information so that it does not need to go back to the control plane for every call
The control planes are pods that also run in the Kubernetes cluster, allowing for better resilience in the event that there is a failure of a single pod in any part of the service mesh
Linkerd
its architecture mirrors Istio’s closely, with an initial focus on simplicity instead of flexibility.
This fact, along with it being a Kubernetes-only solution
While Linkerd v1.x is still supported, and it supports more container platforms than Kubernetes; new features (like blue/green deployments) are focused on v2. primarily.
Istio has the most features and flexibility of any of these three service meshes by far, but remember that flexibility means complexity, so your team needs to be ready for that.
For a minimalistic approach supporting just Kubernetes, Linkerd may be the best choice.
If you want to support a heterogeneous environment that includes both Kubernetes and VMs and do not need the complexity of Istio, then Consul would probably be your best bet.
Migrating between service mesh solutions
Note that service mesh is not as an intrusive transformation as the one from monolithic applications to microservices, or from VMs to Kubernetes-based applications.
Since most meshes use the sidecar model, most services don’t know that they run as a mesh.
Service Mesh is useful for any type of microservices architecture since it helps you control traffic, security, permissions, and observability.
you can start standardizing on Service Mesh in your system design to lay the building blocks and the critical components for large-scale operations
Improving observability into distributed services: For example, If one service in the architecture becomes a bottleneck, the common way to handle it is through re-tries, but that can worsen the bottleneck due to timeouts. With service mesh, you can easily break the circuit to failed services to disable non-functioning replicas and keep the API responsive.
Blue/green deployments: Service mesh allows you to implement Blue/Green deployments to safely rollout new upgrades of the applications without risking service interruption.
First, you expose only a small subset of users to the new version, validate it, then proceed to release it to all instances in Production.
Chaos monkey/ testing in production scenarios: with the ability to inject delays, faults to improve the robustness of deployments
‘Bridge’ / enabler for modernizing legacy applications:If you’re in the throes of modernizing your existing applications to Kubernetes-based microservices, you can use service mesh as a ‘bridge’ while you’re de-composing your apps.
you can use service mesh as a ‘bridge’ while you’re de-composing your apps. You can register your existing applications as ‘services’ in the Istio service catalog and then start migrating them gradually to Kubernetes without changing the mode of communication between services – like a DNS router. This use case is similar to using Service Directory.
API Gateway:If you’re bought into the vision of service mesh and want to start the rollout,you can already have your Operations team start learning the ropes of using service mesh by deploying it simply to measure your API usage.
Service mesh becomes the dashboard for microservices architecture. It’s the place for troubleshooting issues, enforcing traffic policies, rate limits, and testing new code. It’s your hub for monitoring, tracing and controlling the interactions between all services – how they are connected, perform and secured.
https://platform9.com/blog/kubernetes-service-mesh-a-comparison-of-istio-linkerd-and-consul/
HTTP/2 (originally named HTTP/2.0) is a major revision of the HTTP network protocol used by the World Wide Web.
Differences from HTTP 1.1
The proposed changes do not require any changes to how existing web applications work, but new applications can take advantage of new features for increased speed
What is new is how the data is framed and transported between the client and the server.Websites that are efficient minimize the number of requests required to render an entire page by minifying (reducing the amount of code and packing smaller pieces of code into bundles, without reducing its ability to function) resources such as images and scripts. However, minification is not necessarily convenient nor efficient and may still require separate HTTP connections to get the page and the minified resources. HTTP/2 allows the server to "push" content, that is, to respond with data for more queries than the client requested. This allows the server to supply data it knows a web browser will need to render a web page, without waiting for the browser to examine the first response, and without the overhead of an additional request cycle.
https://en.wikipedia.org/wiki/HTTP/2
WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API in Web IDL is being standardized by the W3C.WebSocket is distinct from HTTP. Both protocols are located at layer 7 in the OSI model and depend on TCP at layer 4
Although they are different, RFC 6455 states that WebSocket "is designed to work over HTTP ports 80 and 443 as well as to support HTTP proxies and intermediaries," thus making it compatible with the HTTP protocol. To achieve compatibility, the WebSocket handshake uses the HTTP Upgrade header[1] to change from the HTTP protocol to the WebSocket protocol.
The WebSocket protocol enables interaction between a web browser (or other client application) and a web server with lower overhead than half-duplex alternatives such as HTTP polling, facilitating real-time data transfer from and to the server.
https://en.wikipedia.org/wiki/WebSocket
Why gRPC?
gRPC is a modern open source high performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication.
Bi-directional streaming and integrated auth
Bi-directional streaming and fully integrated pluggable authentication with HTTP/2-based transport
https://grpc.io/
IBM, Google and Lyft give microservices a ride on the Istio Service Mesh
Today, IBM and Google announced the launch of Istio, an open technology that provides a way for developers to seamlessly connect, manage and secure networks of different microservices—regardless of platform, source or vendor.
Istiois the result of a joint collaboration between IBM, Google and Lyft as a means to support traffic flow management, access policy enforcement and the telemetry data aggregation between microservices
Its design, however, is not platform specific. The Istio open source project plan includes support for additional platforms, including CloudFoundry, VMs.
Key Istio features
Automatic zone-aware load balancing and failover for HTTP/1.1, HTTP/2, gRPC, and TCP traffic.
Fine-grained control of traffic behavior with rich routing rules, fault tolerance, and fault injection.
A pluggable policy layer and configuration API supporting access controls, rate limits and quotas.
Automatic metrics, logs and traces for all traffic within a cluster, including cluster ingress and egress.
Secure service-to-service authentication with strong identity assertions between services in a cluster.
How does Istio work?
Improved visibility into the data flowing in and out of apps, without requiring extensive configuration and reprogramming.
Istio converts disparate microservices into an integrated service mesh by introducing programmable routing and a shared management layer. By injecting Envoy proxy servers into the network path between services, Istio provides sophisticated traffic management controls such as load-balancing and fine-grained routing. This routing mesh also enables the extraction of a wealth of metrics about traffic behavior, which can be used to enforce policy decisions such as fine-grained access control and rate limits that operators can configure. Those same metrics are also sent to monitoring systems. This way, it offers improved visibility into the data flowing in and out of apps, without requiring extensive configuration and reprogramming to ensure all parts of an app work together smoothly and securely.
https://developer.ibm.com/dwblog/2017/istio/
Istio
a platform, including APIs that let it integrate into any logging platform, or telemetry or policy system. completely open source service mesh that layers transparently onto existing distributed applications.
lets you successfully, and efficiently run a distributed micro-service architecture, and provides a uniform way to secure, connect, and monitor micro-services.
operators are managing extremely large hybrid and multi-cloud deployments.
Developers must use micro-services to architect for portability Istio helps reduce the complexity of these deployments and eases the strain on your development teams. Istio lets you connect, secure control, and observe services.
What is a service mesh? Istio addresses the challenges developers and operators face as monolithic applications transition towards a distributed micro-service architecture The term service mesh is used to describe the network of micro-services that make up such applications and the interactions between them.
Its requirements can include discovery, load balancing, failure recovery, metrics, and monitoring.
A service mesh also often has more complex operational requirements, like A/B testing, canary releases, rate limiting, access control, and end-to-end authentication.
Why use Istio?
create a network of deployed services with load balancing, service-to-service authentication, monitoring, without any changes in service code.
add Istio support to services by deploying a special sidecar proxy throughout your environment that intercepts all network communication between micro-services, then configure and manage Istio using its control plane functionality
Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic.
Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and fault injection.
Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress.
Secure service-to-service communication in a cluster with a strong identity-based authentication and authorization
Core features
Traffic management
control the flow of traffic and API calls between services
configuration of service-level properties like circuit breakers, timeouts, and retries
tasks like A/B testing, canary rollouts, and staged rollouts with percentage-based traffic splits
Platform support Istio is platform-independent and designed to run in a variety of environments, including those spanning Cloud, on-premise, Kubernetes, Mesos, and more.
You can deploy Istio on Kubernetes, or on Nomad with Consul.
Service deployment on Kubernetes
Services registered with Consul
Services running on individual virtual machines
Architecture
An Istio service mesh is logically split into a data plane and a control plane. The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices along with Mixer, a general-purpose policy and telemetry hub.
The control plane manages and configures the proxies to route traffic. Additionally, the control plane configures Mixers to enforce policies and collect telemetry.
https://istio.io/docs/concepts/what-is-istio/
What is a service mesh?
Istio addresses the challenges developers and operators face as monolithic applications transition towards a distributed microservice architecture.
The term service mesh is used to describe the network of microservices that make up such applications and the interactions between them. As a service mesh grows in size and complexity, it can become harder to understand and manage. Its requirements can include discovery, load balancing, failure recovery, metrics, and monitoring. A service mesh also often has more complex operational requirements, like A/B testing, canary releases, rate limiting, access control, and end-to-end authentication.
Why use Istio?
Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic.
Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and fault injection.
A pluggable policy layer and configuration API supporting access controls, rate limits and quotas.
Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress.
Secure service-to-service communication in a cluster with strong identity-based authentication and authorization.
Security
Istio’s security capabilities free developers to focus on security at the application level. Istio provides the underlying secure communication channel, and manages authentication, authorization, and encryption of service communication at scale. While Istio is platform independent, using it with Kubernetes (or infrastructure) network policies, the benefits are even greater, including the ability to secure pod-to-pod or service-to-service communication at the network and application layers.
https://istio.io/docs/concepts/what-is-istio/
Integrating Calico and Istio to Secure Zero-Trust Networks on Kubernetes
the ZTN model means not allowing access to anyone unless they are authenticated and their request to a specific network resource has been authorized. ZTN builds on the following principles:
Networks should always be assumed to be hostile.
External and internal threats exist on the network at all times.
Network locality is not sufficient for gaining trust.
Every device, user, and workflow should be authenticated and authorized.
Network policies must be dynamic and calculated from as many sources of data as possible
How can Calico and Istio help?
Calico is an open-source project designed to remove the complexities surrounding traditional software-defined networks and securing them through simple policy language in YAML. Calico is compatible with major cloud platforms, such as Kubernetes, OpenStack, Amazon Web Services, and Google Compute Engine.
Calico’s implementation of the Kubernetes Network Policy API enables granular selection and grouping. Policies are configured based on Kubernetes labels.
Istio, another open-source project, resides on the concept of a service mesh by installing an Envoy sidecar proxy as close as possible to an application. This enables management of both the proxy and the application.In the context of security, Istio provides authentication and encryption through mutual TLS—where both client and server use certificates to verify identity—and cryptographic certificates issued to each serviceAccount.
A service mesh is a dedicated infrastructure layer for making service-to-service communication safe, fast, and reliable. If you’re building a cloud native application, you need a service mesh.In practice, the service mesh is typically implemented as an array of lightweight network proxies that are deployed alongside application code, without the application needing to be aware.
WHAT IS A SERVICE MESH?
The concept of the service mesh as a separate layer is tied to the rise of the cloud native application. In the cloud native model, a single application might consist of hundreds of services; each service might have thousands of instances; and each of those instances might be in a constantly-changing state as they are dynamically scheduled by an orchestrator like Kubernetes.
IS THE SERVICE MESH A NETWORKING MODEL?
The service mesh is a networking model that sits at a layer of abstraction above TCP/IP. It assumes that the underlying L3/L4 network is present and capable of delivering bytes from point to point.
Just as the TCP stack abstracts the mechanics of reliably delivering bytes between network endpoints, the service mesh abstracts the mechanics of reliably delivering requests between services.
Like TCP, the service mesh doesn’t care about the actual payload or how it’s encoded.
The application has a high-level goal (“send something from A to B”), and the job of the service mesh, like that of TCP, is to accomplish this goal while handling any failures along the way.
WHAT DOES A SERVICE MESH ACTUALLY DO?
A service mesh like Linkerd manages this complexity with a wide array of powerful techniques: circuit-breaking, latency-aware load balancing, eventually consistent (“advisory”) service discovery, retries, and deadlines.
Linkerd can also initiate and terminate TLS, perform protocol upgrades, dynamically shift traffic, and fail over between datacenters.
WHY IS THE SERVICE MESH NECESSARY?
Consider the typical architecture of a medium-sized web application in the 2000’s: the three-tiered app. In this model, application logic, web serving logic, and storage logic are each a separate layer.The communication between layers, while complex, is limited in scope—there are only two hops, after all. There is no “mesh”, but there is communication logic between hops, handled within the code of each layer.
When this architectural approach was pushed to very high scale, it started to break.
Companies like Google, Netflix, and Twitter, faced with massive traffic requirements
the application layer was split into many services (sometimes called “microservices”), and the tiers became a topology.
In these systems, a generalized communication layer became suddenly relevant, but typically took the form of a “fat client” library—Twitter’s Finagle, Netflix’s Hystrix, and Google’s Stubby being cases in point.
In many ways, libraries like Finagle, Stubby, and Hystrix were the first service meshes.
The cloud native model combines the microservices approach of many small services with two additional factors: containers (e.g. Docker), which provide resource isolation and dependency management, and an orchestration layer (e.g. Kubernetes), which abstracts away the underlying hardware into a homogenous pool.
THE FUTURE OF THE SERVICE MESH
The requirements for serverless computing (e.g. Amazon’s Lambda) fit directly into the service mesh’s model of naming and linking, and form a natural extension of its role in the cloud native ecosystem.
How is Istio related to Cilium? Istio abstracts away a lot of networking specific complexity and provides visibility and control to application teams. We couldn't agree more with the move of networking to Layer 7 and the concept to provide the necessary instruments for efficient operation at the application protocol layer.
What is Cilium?
Cilium comes in the form of a networking plugin and thus integrates at a lower level with the orchestration system. Cilium and Istio share a common goal though, both aim to move visibility and control to the application protocol level (HTTP, gRPC, Kafka, Mongo, ...)
Cilium uses a combination of components to provide this functionality:
An agent written in golang that runs on all nodes to orchestrate everything. This agent is integrated with orchestration systems such as Kubernetes.
A datapath component that utilizes the BPF (Berkley Packet Filter) functionality in the Linux kernel for very efficient networking, policy enforcement, and load balancing functionality.
A set of userspace proxies, one of them is Envoy, to provide application protocol level filtering while we are completing the in-kernel version of this. More on this below.
Can I run Cilium alongside Istio?
It is perfectly fine to run Cilium as a CNI plugin to provide networking, security, and loadbalancing and then deploy Istio on top.
How will Istiobenefit from Cilium?
We are very excited about BPF and how it is changing how security and networking are done with Linux.
Why is the In-kernel proxy faster than not running a proxy at all?
When changing how to approach a problem. Completely new solutions often present themselves. One of them is what we call socket redirect. The in-kernel proxy is capable of having two pods talk to each other directly from socket to socket without ever creating a single TCP packet. This is very similar to having two processes talk to each other using a UNIX domain socket. The difference is that the applications can remain unchanged while using standard TCP sockets.
The difference in the two lines between "No Proxy" and "Cilium In-Kernel" is thus the cost of the TCP/IP stack in the Linux kernel.
How else can Istio and Cilium benefit from each other?
Use of Istio Auth and the concept of identities to enforce the existing Cilium identity concept. This would allow enforcing existing NetworkPolicy with the automatically generated certificates as provided by Istio Auth.
Ability to export telemetry from Cilium to Istio.
Potential to offload Istio Mixer functionality in Cilium
https://cilium.io/blog/istio/
Cilium: Making BPF Easy on Kubernetes for Improved Security, Performance
BPF is basically the ability of an application developer to write a program, load that into the Linux kernel, then run it when certain events happen — when a network packet is being received or a system call is being made, when a certain system function is being called. Then this small program can enforce security policies, collect information and so on. It’s basically making the Linux kernel programmable
I wouldn’t say we compete with Istio, we complement each other,” he said. “Cilium is the ideal data path, data layer, beneath Istio. We provide the best performance possible. If you want to run Istio, we can reduce the overhead and make it minimal. A service mesh runs security policy in a sidecar inside of the application pod. That means if that pod gets compromised, the sidecar is compromised as well. We can provide a safety net outside of that.”
Traditionally you may have had two almost identical servers: one that goes to all users and another with the new features that gets rolled out to only a set of users.
by using GitOps workflows, your canary can be fully controlled through Git.
If something goes wrong and you need to roll back, you can redeploy a stable version all from Git.
An Istio virtual gateway allows you to manage the amount of traffic that goes to both deployments.
With both a GA and a canary deployed, you can continue to iterate on the canary release until it meets expectations and you are able to open it up to 100% of the traffic.
GitOps Workflows for the continuous deployment to Istio:
An engineer fixes the latency issue and cuts a new release by tagging the master branch as 0.2.1
GitHub notifies GCP Container Builder that a new tag has been committed
GCP Container Builder builds the Docker image, tags it as 0.2.1 and pushes it to to Quay.io (this can be any container registry)
Weave Cloud detects the new tag and updates the Canary deployment definition
Weave Cloud commits the Canary deployment definition to GitHub in the cluster repo
Weave Cloud triggers a rolling update of the Canary deployment
Weave Cloud sends a Slack notification that the 0.2.1 patch has been released
Once the Canary is fixed, keep increasing the traffic to it and shift the traffic from the GA deployment by modifying the weight setting and committing those changes to Git.
With each Git push and manifest modification, Weave Cloud detects that the cluster state is out of sync with desired state and will automatically apply the changes.
If you notice that the Canary doesn't behave well under load, revert the changes in Git.Weave Cloud rolls back the weight setting by applying the desired state from Git on the cluster.
You can keep iterating on the canary code until the SLA is on a par with the GA release.
https://www.weave.works/blog/gitops-workflows-for-istio-canary-deployments
The World’s Most Popular Open Source Microservice API Gateway
Invoke serverless functions via APIs.
Kong is a scalable, open source API Layer (also known as an API Gateway, or API Middleware). Kong runs in front of any RESTful API and is extended through Plugins, which provide extra functionality and services beyond the core platform.
https://konghq.com/kong-community-edition/
What are containers?
Containers are an executable unit of software in which application code is packaged, along with its libraries and dependencies, in common ways so that it can be run anywhere, whether it be on desktop, traditional IT, or the cloud.
To do this, containers take advantage of a form of operating system (OS) virtualization in which features of the OS (in the case of the Linux kernel, namely the namespaces and cgroups primitives) are leveraged to both isolate processes and control the amount of CPU, memory, and disk that those processes have access to.
Containers are small, fast, and portable because unlike a virtual machine, containers do not need include a guest OS in every instance and can, instead, simply leverage the features and resources of the host OS.
Containers vs. VMs
Instead of virtualizing the underlying hardware, containers virtualize the operating system (typically Linux) so each individual container contains only the application and its libraries and dependencies. The absence of the guest OS is why containers are so lightweight and, thus, fast and portable.
Benefits
Lightweight: Containers share the machine OS kernel, eliminating the need for a full OS instance per application and making container files small and easy on resources.
Portable and platform independent: Containers carry all their dependencies with them, meaning that software can be written once and then run without needing to be re-configured across laptops, cloud, and on-premises computing environments.
Supports modern development and architecture: such as DevOps, serverless, and microservices—that are built are regular code deployments in small increments.
Containerization
Software needs to be designed and packaged differently in order to take advantage of containers—a process commonly referred to as containerization.When containerizing an application, the process includes an application with its relevant environment variables, configuration files, libraries, and software dependencies. The result is a container image that can then be run on a container platform
Container orchestration
container orchestration emerged as a way managing large volumes of containers throughout their lifecycle, including:
Provisioning
Redundancy
Health monitoring
Resource allocation
Scaling and load balancing
Moving between physical hosts
While many container orchestration platforms (such as Apache Mesos, Nomad, and Docker Swarm) were created to help address these challenges, Kubernetes quickly became the most popular container orchestration platform, and it is the one the majority of the industry has standardized on.
Docker and Kubernetes Kubernetes, created by Google in 2014, is a container orchestration system that manages the creation, operation, and termination of many containers.
Docker turns program source code into containers and then executes them, whereas Kubernetes manages the configuration, deployment, and monitoring of many containers at once (including both Docker containers and others).
Istio, Knative, and the expanding containers ecosystem
the ecosystem of tools and projects designed to harden and expand production use cases continues to grow Istio
As developers leverage containers to build and run microservice architectures, management concerns go beyond the lifecycle considerations of individual containers and into the way that large numbers of small services—often referred to as a “service mesh”—connect with and relate to one another. Istiowas created to make it easier for developers to manage the associated challenges with discovery, traffic, monitoring, security Knative Knative’s big value is its ability to offer container services as serverless functions.
Instead of running all the time and responding when needed (as a server does), a serverless function can “scale to zero,” which means it is not running at all unless it is called upon. This model can save vast amounts of computing power when applied to tens of thousands of containers.
https://www.ibm.com/cloud/learn/containers
What Is AWS Lambda?
AWS Lambda is a compute service that lets you run code without provisioning or managing servers
You can also build serverless applications composed of functions that are triggered by events and automatically deploy them using AWS CodePipeline and AWS CodeBuild.
https://docs.aws.amazon.com/lambda/latest/dg/welcome.html
AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running.
With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app
https://aws.amazon.com/lambda/
Distributed Data Pre-processing using Dask, Amazon ECS and Python
Data Engineers and Data Scientists often use tools from the python ecosystem such as Numpy and Pandas to analyze, transform and visualize their data which are designed to be high performance, intuitive and efficient libraries. Performing such operations on a small dataset in a fast and scalable manner is not challenging as long as the dataset can fit into the memory of a single machine. However, if the dataset is too big and cannot fit into a single machine, Data Engineers may be forced to rewrite their code into more scalable tools such as Spark and SparkMLthat can be computationally supported by a big EMR cluster.
https://towardsdatascience.com/serverless-distributed-data-pre-processing-using-dask-amazon-ecs-and-python-part-1-a6108c728cc4
Build highly scalable applications on a fully managed serverless platform
App Engine enables developers to stay more productive and agile by supporting popular development languages and a wide range of developer tools.
Google App Engine (often referred to as GAE or simply App Engine) is a web framework and cloud computing platform for developing and hosting web applications in Google-managed data centers.
Applications are sandboxed and run across multiple servers.App Engine offers automatic scaling for web applications—as the number of requests increases for an application, App Engine automatically allocates more resources for the web application to handle the additional demand
https://cloud.google.com/appengine/
Your First ServerlessMicroservice on AWS
Intro to Serverless Computing
Serverless computing is a cloud-computing execution model in which the cloud provider acts as the server, dynamically managing the allocation of machine resources. Pricing is based on the actual amount of resources consumed by an application, rather than on pre-purchased units of capacity.
It is a form of utility computing.
Utility computing, or The Computer Utility, is a service provisioning model in which a service provider makes computing resources and infrastructure management available to the customer as needed, and charges them for specific usage rather than a flat rate.
The name "serverless computing" is usedbecause the server management and capacity planning decisions are completely hidden from the developer or operator. The serverless code can be used in conjunction with code deployed in traditional styles, such as microservices. Alternatively, applications can be written to be purely serverless and use no provisioned servers at all
Most, but not all,serverless vendors offer to compute runtimes, also known as function as a service (FaaS) platforms, which execute application logic but do not store data.
Serverless computing is not suited to some computing workloads, such as high-performance computing, because of the resource limits imposed by cloud providers, and also because it would likely be cheaper to bulk-provision the number of servers believed to be required at any given point in time.
Serverless computing allows you to build and run applications and services without thinking about servers. With serverless computing, your application still runs on servers, but all the server management is done by AWS. At the core of serverless computing is AWS Lambda, which lets you run your code without provisioning or managing servers. Learn more about serverless computing by visiting here.
https://aws.amazon.com/lambda/faqs/
Fission is a framework for serverless functions on Kubernetes.
Write short-lived functions in any language, and map them to HTTP requests (or other event triggers). Deploy functions instantly with one command. There are no containers to build, and no Docker registries to manage.
https://fission.io/
FaaS use cases
Web apps, Backends, Data/stream processing, Chatbots, Scheduled tasks, IT Automation
In other words, serverless aims to do exactly what it sounds like — allow applications to be developed without concerns for implementing, tweaking, or scaling a server (at least, to the perspective of a user).
Instead of scaling a monolithic REST server to handle potential load, you can now split the server into a bunch of functions which can be scaled automatically and independently.
OpenFaaS® (Functions as a Service) is a framework for building Serverless functions with Docker and Kubernetes which has first-class support for metrics. Any process can be packaged as a function enabling you to consume a range of web events without repetitive boiler-plate coding.
https://github.com/openfaas/faas
Function as a service (FaaS) is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage application functionalities without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app.
Building an application following this model is one way of achieving a "serverless" architecture, and is typically used when building microservices applications.
Comparison with PaaS application hosting services Serverless computing architectures in which the customer has no direct need to manage resources can also be achieved using Platform as a Service services. These services are, however, typically very different in their implementation architecture, which has some implications for scaling. In most PaaS systems, the system continually runs at least one server process and, even with auto scaling, a number of longer running processes are simply added or removed on the same machine. This means that scalability is a more visible problem to the developer
In a FaaS system, the functions are expected to start within milliseconds in order to allow handling of individual requests. In a PaaS system, by contrast, there is typically an application thread which keeps running for a long period of time and handles multiple requests. This difference is primarily visible in the pricing, where FaaS services charge per execution time of the function whereas PaaS services charge per running time of the thread in which the server application is running.
Use Cases
Use cases for FaaS are associated with "on-demand" functionality that enables the supporting infrastructure to be powered down and not incurring charges when not in use. Examples include data processing (e.g., batch processing, stream processing, ETL), IoT services for Internet-connected devices, mobile applications, and web applications.
https://en.wikipedia.org/wiki/Function_as_a_service
NuclioServerless Functions
Nuclio is an open source serverless platform which allows developers to focus on building and running auto-scaling applications without worrying about managing servers.
https://nuclio.io/
The Serverless Framework – Build applications comprised ofmicroservices that run in response to events, auto-scale for you, and only charge you when they run.
https://github.com/serverless/serverless
What are event-driven programming and serverless computing?
Event-driven programming follows a paradigm in which the flow of an application is determined by events such as user actions, sensor outputs, or messages from other applications or services. Serverless computing refers to a model where the existence of servers is simply hidden from developers. I.e. that even though servers still exist developers are relieved from the need to care about their operation.
What problems does OpenWhisk solve? What can you do with OpenWhisk? OpenWhisk abstracts away all infrastructure and operational concerns, allowing developers to solely focus on coding. As part of that abstraction, you no longer need to worry about peak projections or capacity planning as OpenWhisk scales on demand based on the volume and velocity of your events’ requests
https://openwhisk.apache.org/faq.html
Apache OpenWhisk (Incubating) is a serverless, open source cloud platform that executes functions in response to events at any scale.
https://openwhisk.apache.org/
Microservice architectures have emerged as the preferred way to engineer robust, scalable cloud-native solutions. Microservices hold application logic in small, loosely coupled distributed services, communicating through language-agnostic APIs.
Despite the benefits, microservice-based solutions are hard to build using mainstream cloud technologies, which often require you to control a complex toolchain and build/operations pipeline.
As a result, developers spend too much time dealing with infrastructure and operational complexities like fault-tolerance, load balancing, auto-scaling, and logging features. OpenWhisk can help to power various mobile, web and IoT use-cases; for example, it can enable mobile developers to interface with backend logic running in a cloud without installing server-side middleware or infrastructure.
https://developer.ibm.com/code/open/projects/openwhisk/
Introducing Serverless Composition for IBM Cloud Functions
Mobile Backend Development Serverless could help developers better decouple front and backend development by helping enterprises create serverlessmicroservices with API backends.
Data Processing
Data processing is a very dominant field for serverless, no matter the platform, and that’s also a key use case trend for OpenWhisk. In particular, image processing, or other tasks like noise reduction in sound files, are most common
Cognitive Processing
Similar to the data processing scenario is cognitive processing, where OpenWhiskis usedin conjunction with IBM Watson ML algorithms to perform some cognitive tasks on a multimedia file
Streaming Analytics
He described an integration with Kafka and Bluemix where data posted in Kafka can immediately start being analyzed.
Internet of Things Nauerz pointed to Abilisense as one business that is using Watson and OpenWhisk in combination to manage an IoT messaging platform for people with hearing difficulties.
GreenQ is optimizing waste disposal truck routes in Tel Aviv, estimating that by using serverless architecture to keep track of the truck locations, weigh waste bins, and analyze photos of waste bin contents, they can calculate optimal routes, lower city carbon emissions and create municipal cost savings of up to 50 percent.
Knative as an open source platform. It supports containers, a form of packaged applications that run in cloud environments. Knative runs on top of the Kubernetes container orchestration system, which controls large numbers of containers in a production environment.
Kubernetes is a complex product that needs a lot of configuration to run properly. Developers must log into individual worker nodes to carry out repetitive tasks, such as installing dependencies and configuring networking rules. They must generate configuration files, manage logging and tracing, and write their own CI/CD scripts using tools like Jenkins. Before they can deploy their containers, they have to go through multiple steps to containerize their source code in the first place.
Knative helps developers by hiding many of these tasks, simplifying container-based management and enabling you to concentrate on writing code. It also supports for serverless functions
Knativeconsists of three main components: Build, Serve, and Event.
Build
The Build component of Knative turns source code into cloud-native containers or functions.
Serve
Knative offers a Serve component that runs containers as scalable services. Not only can it scale containers up to run in the thousands, but it can scale them down to the point where there are no instances of the container running at all.
The first is configuration, which lets you create different versions of the same container-based service. Knative lets these different versions run concurrently,
The second feature is service routing.You can use Knative's routing capabilities to send a percentage of user requests to the new version of the service, while still sending most other requests to the old one.
Eve
The Event component of Knative enables different events to trigger their container-based services and functions.
Benefits
Faster iterative development:
Focus on code:
Quick entry to serverless computing:
What challenges does Knative solve?
Knative's benefits can help solve a variety of real-world challenges facing today's developers, including the following:
CI/CD set up
Easier customer rollouts
Serverless
Serverless computing is a relatively new way of deploying code that can help make cloud-native software even more productive. Instead of having a long-running instance of your software waiting for new requests (which means you are paying for idle time), the hosting infrastructure will only bring up instances of your code on an "as-needed" basis.
Knative breaks down the distinction between software services and functions by enabling developers to build and run their containers as both
Istio
Knative's serving component incorporates Istio help manage tiny, container-based software services known as microservices.
Istio provides a routing mechanism that allows services to access each other via URLs in what's known as a service mesh.
It's essentially a switchboard for the vast, complex array of container-based services that can quickly develop in a microservice environment.
Knative uses Istio’s service mesh routing capabilities to route calls between the services that it runs.
Istio manages authentication for service requests and automatic traffic encryption for secure communication between services.
It can also gather detailed metrics about microservice operations to help developers and administrators plan infrastructure optimization.
Serverless Open-Source Frameworks: OpenFaaS, Knative, & More
The main advantage of this technology is the ability to create and run applications without the need for infrastructure management.
In other words, when using a serverless architecture, developers no longer need to allocate resources, scale and maintain servers to run applications, or manage databases and storage systems. Their sole responsibility is to write high-quality code.
There have been many open-source projects for building serverless frameworks (Apache OpenWhisk, IronFunctions, Fn from Oracle, OpenFaaS, Kubeless, Knative, Project Riff, etc).
OpenWhisk, Firecracker & Oracle FN
Apache OpenWhisk is an open cloud platform for serverless computing that uses cloud computing resources as services. Compared to other open-source projects (Fission, Kubeless, IronFunctions), Apache OpenWhisk is characterized by a large codebase, high-quality features, and the number of contributors. However, the overly large tools for this platform (CouchDB, Kafka, Nginx, Redis, and Zookeeper) cause difficulties for developers. In addition, this platform is imperfect in terms of security
Firecracker is a virtualization technology introduced by Amazon.Firecracker offers lightweight virtual machines called micro VMs, which use hardware-based virtualization technologies for their full isolation while at the same time providing performance and flexibility at the level of conventional containers.The project was developed at Amazon Web Services to improve the performance and efficiency of AWS Lambda and AWS Fargate platforms.
Oracle Fn is an open-server serverless platform that provides an additional level of abstraction for cloud systems to allow for Functions as Services (FaaS). As in other open platforms in Oracle Fn, the developer implements the logic at the level of individual functions. Unlike existing commercial FaaS platforms, such as Amazon AWS Lambda, Google Cloud Functions, and Microsoft Azure Functions, Oracle’s solution is positioned as having no vendor lock-in.
Kubeless is an infrastructure that supports the deployment of serverless functions in your cluster and enables us to execute both HTTP and event switches in your Python, Node.js, or Ruby code. Kubeless is a platform that is built using Kubernetes’ core functionality, such as deployment, services, configuration cards (ConfigMaps), and so on.
Fission is an open-source platform that provides a serverless architecture over Kubernetes. One of the advantages of Fission is that it takes care of most of the tasks of automatically scaling resources in Kubernetes, freeing you from manual resource management. The second advantage of Fission is that you are not tied to one provider and can move freely from one to another, provided that they support Kubernetes clusters (and any other specific requirements that your application may have).
Main Benefits of Using OpenFaaS and Knative
OpenFaaS and Knative are publicly available and free open-source environments for creating and hosting serverless functions
These platforms allow you to:
Reduce idle resources.
Quickly process data.
Interconnect with other services.
Balance load with intensive processing of a large number of requests.
How to Build and Deploy Serverless Functions With OpenFaaS
OpenFaaS is a Cloud Native serverless framework and therefore can be deployed by a user on many different cloud platforms as well as bare-metal servers.The main goal of OpenFaaS is to simplify serverless functions with Docker containers, allowing you to run complex and flexible infrastructures.There are installation options for Kubernetes and Docker Swarm.Docker is not the only runtime available in Kubernetes, so others can be used.
Function Watchdog
Almost any code can be converted to an OpenFaaS function. If your use case doesn’t fit one of the supported language templates then you can create your own OpenFaaS template using the watchdog to relay the HTTP requests to your code inside the container.
all developed functions, microservices, and products are stored in the Docker container, which serves as the main OpenFaaS platform for developers and sysadmins to develop, deploy, and run serverless applications with containers.
The Main Points for Installation of OpenFaaS on Docker
You can install OpenFaaS to any Kubernetes cluster, whether using a local environment, your own datacenter, or a managed cloud service such as AWS EKS
For running locally, the maintainers recommend using the KinD (Kubernetes in Docker) or k3d (k3s in Docker) project. Other options like Minikube and microk8s are also available.
Pros and Cons of OpenFaaS
In other words, OpenFaaS allows you to run code in any programming language anytime and anywhere.
OpenFaaS uses container images for functions
Each function replica runs within a container, and is built into a Docker image.
There is an option to avoid cold starts by having a minimum level of scale such as 20/100 or 1/5
Scaling to zero is optional, but if used in production, you can expect just under a two-second cold-start for the first invocation
The queue-worker enables asynchronous invocation, so if you do scale to zero, you can decouple any cold-start from the user
Deploying and Running Functions With Knative
Knative allows you to develop and deploy container-based server applications that you can easily port between cloud providers
Building
The Building component of Knative is responsible for ensuring that container assemblies in the cluster are launched from the source code. This component works on the basis of existing Kubernetes primitives and also extends them.
Eventing
The Eventing component of Knative is responsible for universal subscription, delivery, and event management as well as the creation of communication between loosely coupled architecture components. In addition, this component allows you to scale the load on the server.
Serving
The main objective of the Serving component is to support the deployment of serverless applications and features, automatic scaling from scratch, routing and network programming for Istio components, and snapshots of the deployed code and configurations. Knative uses Kubernetes as the orchestrator, and Istio performs the function of query routing and advanced load balancing.
Example of the Simplest Functions With Knative
Your choice will depend on your given skills and experience with various services including Istio, Gloo, Ambassador, Google, and especially Kubernetes Engine, IBM Cloud, Microsoft Azure Kubernetes Service, Minikube, and Gardener.
Pros and Cons of Knative
Like OpenFaaS, Knative allows you to create serverless environments using containers. This in turn allows you to get a local event-based architecture in which there are no restrictions imposed by public cloud services.
Both OpenFaaS and Knative let you automate the container assembly process, which provides automatic scaling. Because of this, the capacity for serverless functions is based on predefined threshold values and event-processing mechanisms.
In addition, both OpenFaaS and Knative allow you to create applications internally, in the cloud, or in a third-party data center. This means that you are not tied to any one cloud provider.
One main drawback of Knative is the need to independently manage container infrastructure. Simply put, Knative is not aimed at end-users.
It is worth noting that these platforms can not be easily compared because they are designed for different tasks.
From the point of view of configuration and maintenance, OpenFaas is simpler. With OpenFaas, there is no need to install all components separately as with Knative, and you don’t have to clear previous settings and resources for new developments if the required components have already been installed.
