Cilium 1.7: Hubble UI, Cluster-wide Network Policies, eBPF-based Direct Server Return, TLS visibility, New eBPF Go Library, ...
We are excited to announce the Cilium 1.7 release. A total of 1551 commits have been contributed by a community of 141 developers, many of whom made their first contributions this cycle. Cilium 1.7 brings with it a trove of exciting new features:
- Hubble: We've heard lots of positive feedback on Hubble since the announcement. To make cluster connectivity easier to visualize and debug, we've released a new Hubble UI as open source so you can tweak and extend it too! We've also been working on various improvements to the core Hubble implementation, including better correlation between network flow data and Kubernetes resources. (More details)
- Cilium Cluster-wide Network Policies: The 1.7 release brings the much-anticipated Cluster-wide CNP feature. This allows users to apply baseline network policies which apply to pods across the cluster, regardless of the namespace that the pod resides in. (More details)
- Kube-proxy replacement with Direct Server Return: As presented at Kubecon US 2019 and FOSDEM 2020, this release rounds out the full service feature set for replacing kube-proxy and additionally adds support for Direct Server Return (DSR). This further improves the latency and performance of the kube-proxy replacement in Cilium. Moreover, our kube-proxy replacement comes out of beta and is automatically enabled in environments with newer kernels. (More details)
- Extending L7 policies with TLS introspection: We've added support to Cilium to configure Envoy TLS certificates via Kubernetes resources or local files. This allows Cilium to transparently observe HTTP calls and enforce API-aware policies on TLS-encrypted sessions. (More details)
- L7 visibility annotations for pods: Previously, to gain L7 visibility into traffic in the cluster, users would need to write network policies that put pods into a default deny posture. Visibility annotations now allow users to gain L7 visibility into network traffic first, then subsequently craft full network policies using these insights. (More details)
- Pure Go eBPF library: This is the first Cilium release to begin using the pure Go eBPF library co-written between the Cilium community and CloudFlare. This streamlined library already allowed Cilium to jettison CGo, improving performance and reducing binary sizes. (More details)
- Improvements to scalability through Kubernetes EndpointSlice support and Cilium agent improvements, development on upstream Linux, running our testing environment on managed Kubernetes, distributing Cilium via Helm repositories... and much more! For more highlights, see the 1.7 Release Highlights.
What is Cilium?
Cilium is open source software for transparently providing and securing the network and API connectivity between application services deployed using Linux container management platforms such as Kubernetes.
At the foundation of Cilium is a new Linux kernel technology called eBPF, which enables the dynamic insertion of powerful security, visibility, and networking control logic within Linux itself. eBPF is utilized to provide functionality such as multi-cluster routing, load balancing to replace kube-proxy, transparent encryption as well as network and service security. Besides providing traditional network level security, the flexibility of eBPF enables security with the context of application protocols and DNS requests/responses. Cilium is tightly integrated with Envoy and provides an extension framework based on Go. Because eBPF runs inside the Linux kernel, all Cilium functionality can be applied without any changes to the application code or container configuration.
See the section Introduction to Cilium for a more detailed general introduction to Cilium.
New USERS.md file: Who is using Cilium?
* N: Adobe, Inc. D: Adobe's Project Ethos uses Cilium for multi-tenant, multi-cloud clusters U: L3/L4/L7 policies L: https://youtu.be/39FLsSc2P-Y * N: CENGN - Centre of Excellence in Next Generation Networks D: CENGN is using Cilium in multiple clusters including production and development clusters (self-hosted k8s, On-premises) U: L3/L4/L7 network policies, Monitoring via Prometheus metrics & Hubble L: https://www.youtube.com/watch?v=yXm7yZE2rk4 Q: @rmaika @mohahmed13 * N: Datadog D: Datadog is using Cilium in AWS (self-hosted k8s) U: ENI Networking, Service load-balancing, Encryption Q: @lbernail, @roboll [...]
Sharing experiences and learning from other users is essential. We are frequently asked who is using a particular feature of Cilium to get in contact with other users to share experiences and best-practices. While the Cilium Slack community allows users to get in touch, it can be challenging to find users of a particular feature quickly.
If you are using Cilium, please consider adding yourself as a user with a quick description of your use case by opening a pull request to this file and adding a section describing your usage of Cilium. If you are open to others contacting you about your use of Cilium on Slack, add your Slack nick as well.
Contributed by Sebastian Wicki and Sergey Generalov
The development cycle of Cilium 1.7 coincided with the a first preview release of Hubble -- an observability tool specifically designed for Cilium. Hubble is able to obtain deep visibility into the network traffic of Kubernetes application and services by tapping into Cilium's eBPF data path. This information can then be queried via Hubble CLI and UI, for example for interactive troubleshooting of DNS issues. For monitoring, Hubble provides an extensible metrics framework which integrates nicely into Prometheus and Grafana. For more information please refer to the tutorial on setting up Hubble Metrics with Grafana.
Several features in Cilium 1.7 have been added with Hubble in mind: The L7 visibility annotations for pods for example allows Hubble to extract application-layer information from DNS and HTTP traffic. The Cilium API has also been extended in this release to allow Hubble to annotate the observed network flows with additional meta-data, such as mapping Kubernetes ClusterIPs to their respective service names.
Hubble UI enables zero-effort automatic discovery of the service dependency graph for Kubernetes Clusters at L3/L4 and even L7, allowing user-friendly visualization and filtering of those dataflows as a Service Map. First presented during the Hubble announcement, we provided users with a preview release docker image, allowing everyone to try Hubble Service Map in a Minikube while working on open sourcing the code itself.
We are happy to announce, Hubble UI code is now open sourced and available under Cilium's GitHub organization: https://github.com/cilium/hubble-ui
During Cilium 1.7 development cycle several performance improvements were made to Hubble UI to work better in small multi-node clusters; however we still consider Hubble UI in preview release stage and encourage the community to provide feedback on Hubble Github page or Cilium Slack #hubble channel.
Cilium Cluster-wide Network Policies
Contributed by Deepesh Pathak and André Martins
This release introduces Cilium Cluster-wide Network Policies (CCNP). Prior to Cilium 1.7, all Cilium network policies were namespaced, so there was no easy way to configure a baseline policy that applies across the entire cluster. Cluster-wide policies streamline the application of a standard default posture by allowing the cluster maintainer to apply a single policy which applies to pods in all namespaces, regardless of the policies that exist in individual namespaces. Cluster-wide policies are essential in various cases, such as:
- Automatically applying a default-deny policy to all namespaces as they're created;
- Allowing requests to a baseline set of allowed destinations like kube-dns, DNS destinations used by all apps, or known IP ranges;
- Reducing management overhead of network policies in high-scale environments.
CiliumClusterwideNetworkPolicy resource specification is the same as that
CiliumNetworkPolicy CRD with the only difference in the scope of
the policy, denoted by the "kind" field in the YAML. Resource-based Access
Control (RBAC) can be defined separately for CCNP so users modifying policies
in one namespace won't roll back the baseline policies. The policy example
below grants any pod with the label
group: my-app in the entire cluster the
privilege to perform DNS requests via kube-dns:
apiVersion: 'cilium.io/v2' kind: CiliumClusterwideNetworkPolicy description: 'Default deny and allow egress to kube-dns pod.' metadata: name: 'clusterwide-policy-example' spec: endpointSelector: matchLabels: group: my-app egress: - toEndpoints: - matchLabels: 'k8s:io.kubernetes.pod.namespace': kube-system k8s-app: kube-dns toPorts: - ports: - port: '53'
With the new Kubernetes CRD introduced in this release for cluster-wide policy, it's now simpler to create network policies which apply to the entire cluster.
Kube-proxy replacement with Direct Server Return
Contributed by Martynas Pumputis, Daniel Borkmann, Sebastian Wicki and André Martins
This release brings many improvements and adds additional features to Cilium's
kube-proxy replacement in eBPF, first introduced in Cilium
v1.6. The eBPF-based kube-proxy
replacement implements handling of Kubernetes services of type
Kube-proxy replacement in eBPF has many benefits when compared to the vanilla kube-proxy of Kubernetes, such as better performance, reliability and debuggability. See Kubernetes without kube-proxy for a quick-start guide and advanced configuration options. In addition, see Kubecon US 2019 and FOSDEM 2020 for implementation details and performance benchmarks.
In this release, Cilium's eBPF-based kube-proxy replacement has been stabilized and as a result moved from beta status to general availability. New Cilium deployments via Helm transparently enable the kube-proxy replacement components by default if supported by the underlying Linux kernel, meaning, even if run alongside a kube-proxy environment, users still benefit from the eBPF-based data path optimizations.
Check out our 2 minute Cilium demo on running Kubernetes without kube-proxy and netfilter/iptables being compiled out of the kernel:
Direct Server Return
When accessing a Kubernetes service from outside via NodePort, ExternalIPs or LoadBalancer, a Kubernetes worker node might redirect the request to another node. This happens when a service endpoint runs on a different node than the request was sent to. Before the redirect, the request is SNAT'd, which means that the backend won't see the source IP address of a client. Also, the reply will be sent through the initial node back to the client, which introduces additional latency.
To avoid that, Kubernetes offers
externalTrafficPolicy=Local which helps to
preserve the client source IP address by dropping a request to a service if a
receiving node does not run any service endpoint. However, this complicates
load-balancer implementations, and can lead to uneven load balancing.
To address the problem, we have implemented Direct Server Return for Kubernetes services with the help of eBPF. This not only preserves the client source IP address, but also allows us to avoid an extra hop when sending a reply back to the client as shown in the figures below:
Support for Kubernetes Services with External IPs
We have added support for services which are exposed with External IPs.
For each incoming packet with the destination IP set to one of those
the traffic will be redirected to one of the pods being backed by that service.
apiVersion: v1 kind: Service metadata: name: external-service namespace: kube-system spec: ports: - name: service-port protocol: TCP port: 8080 externalIPs: - 10.0.0.1 - 10.0.0.2 - 10.0.0.3
Optimizations for Service Endpoint Selection
Accessing a Kubernetes service from a Cilium-managed node via ClusterIP,
NodePort, ExternalIPs or LoadBalancer is handled through socket-based
load balancing in eBPF. This means that instead of performing slower DNAT on
the packet itself in lower layers of the stack, the backend is selected once,
for example, during the TCP
connect(2) syscall where the kernel proceeds with
directly connecting to the service' backend address.
For any NodePort and ExternalIPs service, we implemented an optimization where the backend can directly be selected on the local node for traffic from the host namespace or from Cilium-managed Kubernetes pods. This is different compared to the regular kube-proxy implementation that would need an additional hop in the network in order to first reach the related node address of the service which in turn would then forward the request to a remote backend in the worst case.
This latency improvement of saving an additional hop for packets is transparent to applications and made possible in Cilium given every Cilium-managed node has a global view of Kubernetes services and their backends, and internal knowledge of security identities of remote Cilium-managed nodes.
- Support for services of the
LoadBalancertype was added, and we have successfully tested the kube-proxy replacement with MetalLB.
- The kube-proxy replacement in Cilium now fully supports the aforementioned
externalTrafficPolicysetting for both
LoadBalancerservices. For services with the
Localtraffic policy, requests to nodes without any local service endpoints will be dropped rather than forwarded, therefore avoiding any unwanted additional hops.
- To accommodate external load balancers which have to learn about the
availability of service endpoints on individual nodes, this release of Cilium
now also supports Kubernetes'
healthCheckNodePortfield. The Cilium user-space agent now serves a service health check for each Kubernetes service of type
- For each NodePort service kube-proxy opens a socket in the host namespace,
and binds the NodePort port to it in order to prevent other applications
from reuse. For a large number of services, this adds a lot of resource
overhead in the kernel, potentially allocating and binding thousands of sockets.
Cilium's kube-proxy replacement avoids this issue entirely, does not allocate
a single socket, and instead uses the eBPF
bind(2)hook through its socket-based load balancing mechanism to consult its eBPF service map and reject an application's
bind(2)request on these ports with an error code.
- Cilium's socket-based load balancing in eBPF now also supports IPv4-in-IPv6
service address translation. This allows IPv4 service backend selection out
sendmsg(2)hook of IPv6-only applications. This is often enabled by default in language runtimes such as in case of Java.
- We have developed an extensive kube-proxy compatibility test suite with over 350 test cases which now runs as part of our Cilium CI infrastructure to ensure same semantics for our eBPF kube-proxy replacement.
TLS visibility for L7 policies (beta)
Contributed by Jarno Rajahalme
Cilium Network Policy (CNP) specification has two new experimental extensions (subject to change in coming releases) for policy enforcement on TLS protected HTTP connections. Firstly, the port-level policy rule definition is extended with TLS contexts, one for terminating the client-initiated TLS connection in the Cilium host proxy, and the other for originating TLS for the upstream proxy connection. In order for this proxy TLS interception to work, the client pod must be configured with your local CA certs that are also used to create the TLS secrets for the port rule. A new Getting Started Guide has step-by-step instructions for this.
Secondly, the HTTP rule level is extended with new header matches that can
perform header manipulations on mismatching headers. The header value being
matched can be sourced from a Kubernetes Secret, so no secret information needs to be
specified directly in the CNP. The supported mismatch actions include
REPLACE. These allow a policy to either just inspect and log
incorrect header values, or replace incorrect values with the correct
ones. Applying the
REPLACE mismatch action on an client authorization header
makes it possible to never expose secret tokens to application pods.
Using these new facilities with
toFQDNs rules allows not only limiting TLS
traffic to external services on specific domain names, but also enforcing and
access logging the HTTP metadata (path, method, headers, etc.), preventing
potentially malicious exfiltration of data, for example.
L7 protocol visibility via pod annotations
Contributed by Ian Vernon and Joe Stringer
When users run Cilium as the CNI, by default the only visibility that is
available via tools like Hubble or
cilium monitor is information at layers
2, 3 and 4 as individual packets pass through the Cilium eBPF data path. Users
can apply Layer 7 Policy to add API-aware visibility and enforcement on the
network streams in the cluster, but for any visibility to be effective without
dropping any other traffic, users would need to craft full policies for the
selected endpoints to allow all expected traffic to/from those endpoints.
To allow users to gain introspection into the API calls being made without
having to craft full policies for their endpoints, we've added support for
proxy visibility annotations. Users can annotate the pod indicating the
direction, port and protocols active on the port, then Cilium gathers and
distributes information about API calls being made to other tools like Hubble.
In the image below, the
tiefighter is posting API requests to
v1/request-landing on the
deathstar... I wonder what that's about!
The policy documentation describes how to use these pod annotations in more detail.
Pure Go eBPF library
Contributed by Joe Stringer
During the Linux Plumbers 2019 eBPF track, Cilium core developers and Cloudflare engineers co-presented a proposal for a pure Go eBPF library which would aim to solve eBPF kernel interactions for long-lived Go-based daemons like Cilium or Cloudflare's L4 load balancer, without the need to pull in CGo.
This effort is now well under way, and in Cilium 1.7 the ring-buffer used for sending messages
from the eBPF data path to the user space
cilium-agent process has been converted over from the
previous CGo implementation to a faster, more efficient implementation with the help of the new
There are various other eBPF libraries in the wild, for example, libbpf or libbcc. While the former represents the canonical implementation, lives in the Linux kernel tree and is suited for C/C++-based applications, it cannot be used in pure Go context. Similarly, that is the case for the latter which has a focus on tracing, wraps libbpf and even LLVM's eBPF backend. Other Go-based libraries in this area depend on CGo, complicating builds and having expensive context switches between C and Go environment as a result to just name a few.
With the start of a generic, pure Go eBPF library, we aim to solve both Cilium and Cloudflare's production needs for orchestrating eBPF, and hope to also enable a much larger Go community to interact with the kernel's eBPF subsystem more easily. The main goals are to cover networking use-cases, to minimise external dependencies, to solve common problems and to have a well-tested and highly testable eBPF library in pure Go that can be used in production.
Aside from the initial eBPF map, program and ring-buffer interactions, the development on the library continues with recent extensions to support the BPF Type Format (BTF) and initial proposals for static data substitution to support templated eBPF programs which only need to be compiled once.
To learn more about the eBPF library, visit the project under Cilium's GitHub organization: https://github.com/cilium/ebpf
Kubernetes EndpointSlice support
Contributed by André Martins
In order to have better scalability for a large number of endpoints backed by a service, Kubernetes 1.16 has introduced EndpointSlice. Since Kubernetes 1.17 this API has been marked as beta and enabled by default. Although the API endpoint is enabled by default, the controller that manages Endpoint Slices is not and one needs to follow the guide here to enable that controller and make use of this new type.
Cilium 1.7 introduces a new flag,
enable-k8s-endpoint-slice, which is enabled
by default and will automatically detect if Endpoint Slices are available in the
cluster and use those to perform all the service translations in eBPF. Setting
this flag to false will fallback to using the v1/Endpoints types available in
Contributed by Ian Vernon and André Martins
CNP node status
As part of the CiliumNetworkPolicy (CNP) scalability improvements, Cilium
introduces a new flag:
When a new CNP is created in the cluster, all Cilium agents will receive an
event from Kubernetes and, as soon as they enforce the policy in the data path,
each one of them will update its status in the status field of the CNP. For a
large number of nodes this can cause high CPU usage in kube-apiserver as for
each update received from each Cilium agent a new Kubernetes event needs to be
sent to all remaining nodes. In previous releases, we have supported entirely
disabling this feature using the
--disable-cnp-status-updates flag; however
this prevents users from understanding the enforcement status of the CNPs.
enable-k8s-event-handover enabled, the implementation will behave
slightly differently: Instead of updating its status field in the CNP, each Cilium
agent will update its status into the KVstore. Cilium Operator will then watch
for all of those updates for each CNP from the KVstore and perform incremental
updates with all Cilium agent status to each CNP into Kubernetes. In the end,
the CNP status will always be present in Kubernetes, but the way it will be
more efficiently populated.
As Cilium does no longer depend on container runtimes, all of the container runtimes' dependencies were removed from Cilium causing the cilium-agent binary size to drop from 97M to 74M.
Cilium 1.7 is compiled with Golang 1.13 which allows a lot of memory optimizations in the runtime and also decreases the memory footprint of Cilium.
Linux kernel changes
Contributed by Daniel Borkmann
During the Cilium 1.7 development window, we've also worked on a number of improvements to the Linux kernel's eBPF subsystem which we co-maintain. The selected changes highlighted below are generic for all eBPF users, but have been implemented in the context of Cilium's and Hubble's eBPF needs. They are part of the just released 5.5 kernel.
Live-Patching of eBPF programs
Given that Cilium's data path needs to support a wide range of Linux kernels, that is, from 4.9 up to the very latest kernel release. A lot of the eBPF data path functionality has been split into eBPF tail calls in particular for the case of having Cilium's eBPF kube-proxy replacement enabled. Aside from reducing verifier complexity on older kernels, the use of eBPF tail calls also allows for atomically replacing pod-specific eBPF features without service disruption on a live system.
The x86-64 eBPF JIT compiler in the kernel originally mapped eBPF tail calls into
an indirect jump, meaning, the target eBPF program address is loaded from the
BPF tail call map from a given index into register
%rax followed by a
Due to the various speculative execution flaws on modern CPUs, we later changed
the eBPF JIT to emit retpolines
instead at the cost of performance. Given also that all major compilers have
adapted this technique, the Linux kernel community in general has since been
obsessed with avoiding indirect calls in fast-path code whenever possible. For
example, a small improvement
to turn eBPF map-related helpers into direct calls showed a 14% performance gain.
Now, for the 5.5 kernel, we implemented tracking of eBPF tail call map indices in the verifier and if the latter determines that a given index is constant from all program paths, as in the vast majority of cases in Cilium's eBPF programs, we can emit a direct jump. Once the tail called programs are updated, the eBPF JIT image is patched to directly jump to the new location.
To demonstrate this technique, the below example eBPF program implements an eBPF tail
call jump to the constant map index
0: (b7) r3 = 0 1: (18) r2 = map[id:526] 3: (85) call bpf_tail_call#12 4: (b7) r0 = 1 5: (95) exit
The x86-64 eBPF JITed program would emit a retpoline on older kernels (marked in bold):
0xffffffffc076e55c: [...] _ 19: xor %edx,%edx |_ index (r3 = 0) 1b: movabs $0xffff88d95cc82600,%rsi |_map (r2 = map[id:526]) 25: mov %edx,%edx | index >= array->map.max_entries check 27: cmp %edx,0x24(%rsi) | 2a: jbe 0x0000000000000066 |_ 2c: mov -0x224(%rbp),%eax | tail call limit check 32: cmp $0x20,%eax | 35: ja 0x0000000000000066 | 37: add $0x1,%eax | 3a: mov %eax,-0x224(%rbp) |_ 40: mov 0xd0(%rsi,%rdx,8),%rax |_prog = array->ptrs[index] 48: test %rax,%rax | prog == NULL check 4b: je 0x0000000000000066 |_ 4d: mov 0x30(%rax),%rax | goto *(prog->bpf_func + prologue_size) 51: add $0x19,%rax | 55: callq 0x0000000000000061 | retpoline for indirect jump 5a: pause | 5c: lfence | 5f: jmp 0x000000000000005a | 61: mov %rax,(%rsp) | 65: retq |_ 66: mov $0x1,%eax (next instruction, r0 = 1) [...]
For 5.5 or later kernels, the same program would get optimized into a direct jump (marked in bold) without the need for a retpoline:
0xffffffffc08e8930: [...] _ 19: xor %edx,%edx |_ index (r3 = 0) 1b: movabs $0xffff9d8afd74c000,%rsi |_map (r2 = map[id:526]) 25: mov -0x224(%rbp),%eax | tail call limit check 2b: cmp $0x20,%eax | 2e: ja 0x000000000000003e | 30: add $0x1,%eax | 33: mov %eax,-0x224(%rbp) |_ 39: jmpq 0xfffffffffffd1785 |_[direct] goto *(prog->bpf_func + prologue_size) 3e: mov $0x1,%eax (next instruction, r0 = 1) [...]
Upon program update, the instruction on address
39, that is
would be live-updated with the address of the new target eBPF program. Similarly,
if the target eBPF program would get deleted from the tail call map, then the
is patched into a same-sized
nop instruction in order to allow a fall-through:
0xffffffffc08e8930: [...] _ 19: xor %edx,%edx |_ index (r3 = 0) 1b: movabs $0xffff9d8afd74c000,%rsi |_map (r2 = map[id:526]) 25: mov -0x224(%rbp),%eax | 2b: cmp $0x20,%eax . 2e: ja 0x000000000000003e . 30: add $0x1,%eax . 33: mov %eax,-0x224(%rbp) |_ 39: nopl 0x0(%rax,%rax,1) |_ fall-through nop 3e: mov $0x1,%eax (next instruction, r0 = 1) [...]
Thus, instead of redirecting speculation into the
pause/lfence loop, the kernel
eliminates the need to potentially perform any speculation for the jump given the
direct address and can therefore execute the generated eBPF code more efficiently.
See the merged patch set for further information.
Safe and multi-architecture supported eBPF probe helpers for tracing
Cilium's eBPF data path is able to export tracing information at various aggregation levels through a high-performance, customizable ring-buffer to its user space agent. Hubble, which builds on top of Cilium and implements a fully distributed networking and security observability platform, is then able to provide deep visibility into the communication and behavior of services as well as the networking infrastructure. Additionally, Hubble also enriches the gathered information through eBPF-based kernel tracing.
Since multi-architecture support is in development for the next Cilium 1.8 release, we have implemented a new set of eBPF helpers in the kernel which allow for safe and multi-architecture eBPF-based memory probing.
The current set of
bpf_probe_read_str() eBPF helpers have
several downsides: while generally safe due to disabling page-faulting, on x86-64,
these helpers can still trigger a kernel warning when attempting to probe user memory on a non-canonical user access address. This can
be problematic since the non-canonical address range is often used in user space
applications for tagged pointers.
Another severe downside is that the two mentioned eBPF helpers are incompatible for non-x86
based architectures as they assume usage for probing memory access for kernel space
addresses as well as user space addresses. However, use for both cases will attempt
to always access kernel space address space given access is performed under
and, while x86 has a non-overlapping address space, other architectures do not,
meaning, kernel pointer and user pointer can have the same address value.
Therefore, we have added the set of
bpf_probe_read_kernel_str() eBPF helpers to
the kernel for strict access under either
KERNEL_DS in their allowed
range and without the possibility to trigger a non-canonical access warning in the
See the merged patch set for further information.
Supporting Cilium testing on managed Kubernetes offerings
Contributed by Maciej Kwiek and Ray Bejjani
Running Cilium end-to-end tests has always been a bit painful because of tight coupling between our testing framework and assumptions about the cluster that tests are being run on. This release, we've done a lot of work to make developers' lives easier.
Cilium 1.7 gives Cilium developers means to run most of our test suite on Kubernetes clusters managed by providers such as GKE. Developers are also able to run only selected tests on our CI GKE clusters, which eases the burden of managing local clusters and allows for more test-driven development thanks to faster cluster provisioning.
Helm 3 and Helm repository
Contributed by Arthur Evstifeev and Joe Stringer
Helm3 was recently released, simplifying the use of helm charts repositories and allowing installation without requiring Tiller to be installed in your cluster. Extending upon the helm template support introduced initially in Cilium 1.6, from this release onwards Cilium will be available via a helm repository residing at https://helm.cilium.io. All Cilium guides have been updated to use Helm3 syntax. To use this repository for installation:
Example: Configuring Cilium for GKE
helm repo add cilium https://helm.cilium.io/ helm install cilium cilium/cilium \ --namespace cilium \ --set global.cni.binPath=/home/kubernetes/bin \ --set global.nodeinit.enabled=true \ --set nodeinit.reconfigureKubelet=true \ --set nodeinit.removeCbrBridge=true
1.7 Release Highlights
- Enhancements to kube-proxy replacement in eBPF
- New Direct Server Return (DSR) mode for better latency and client source IP preservation
- Kubernetes ExternalIPs and LoadBalancer service support
- NodePort services health check support added
- Handling of
- IPv4-in-IPv6 support for socket-based load balancing
- Optimized endpoint selection for socket-based load balancing
- Various SNAT optimizations and better port collision handling
- Efficient detection of NodePort and ExternalIPs port reuse via bind hook
- New feature probe mode by default for new deployments
- Extensive kube-proxy compatibility test suite for Cilium CI
- TLS visibility policies (beta)
- L7 visibility via pod annotations
- Support handling remote nodes as separate identity
- Improve handling of DNS timeouts with FQDN policy
- Validated with Kubernetes 1.17
- Support dual-stack mode
- Support for EndpointSlices
- Better CRD validation for Cilium resources
- More efficient ring-buffer communication between eBPF and cilium-agent
- Support more flexible aggregation of connection events
- Better bounding on dumping large maps
- Improved handling of large CIDR policies
- Detect feature support using bpftool
- Support for forwarding ICMP fragmentation needed messages via agent option
- Support binding to NodePorts
- Scalability and Resource consumption
CiliumNetworkPolicystatus reporting to KVstore and cilium-operator
- Optimize footprint using Go 1.13
- Remove dependency on container runtime (CRI).
- Improved information in the
- Command completion for zsh
- Improved information in the
- Use Helm 3 for deployment instructions
- Add a dedicated repository for Cilium helm charts.
- Various fixes for chaining and managed Kubernetes guides
- Support for 1.4.3
- Kernel changes
- Live-patching eBPF programs
- Multi-architecture support improvements
- Hubble graphical user interface
- Correlate additional metadata with network flows
- Continuous Integration / Testing
- Support for running the Cilium CI in GKE and EKS managed clusters
See the Changelog for full notes on changes during the Cilium 1.7 development cycle.
New to Cilium? Follow one of the Getting Started Guides.
As usual, follow the upgrade guide to upgrade your Cilium deployment. Feel free to ping us on Slack.
- Release Notes & Binaries: 1.7.0
- Container image: