On May 25 2019, a security relevant bug has been reported to us via the documented security disclosure channel. Thanks to l14n for the excellent bug report! It was soon identified that multiple vendors are affected by this vulnerability. This lead to an embargo period which is being lifted today.

The bug allows, under certain circumstances, to bypass network security policies. See below for details on the vulnerability and the mitigation.

Who is affected?: Users operating Cilium in encapsulation mode (VXLAN or Geneve) while hosting untrusted workloads with an egress policy that allows pods to emit UDP encapsulation traffic to other worker nodes.

The vulnerability is being tracked by CVE-2019-13119.

We are releasing Cilium 1.5.4, 1.4.5, and 1.3.7 to fix the security vulnerability.

As with everything we do, we are fully transparent. As it becomes obvious that a simple resolution in this matter is not possible, we follow open source best practices and choose a public forum for the sake of transparency.

It was brought to our attention that some of the new eBPF code committed to the Calico repository is violating the license of source code in the Cilium repository.

The original report called out suspiciously similar code in both repositories. This by itself is of course not a problem if the open source licenses involved are respected. This includes, among other things, attribution and restrictions regarding the rights to re-license.

Upon closer inspection, it was identified that source code has been copied from the Cilium repository, modified to create derivative work, and then committed (commit) to the Calico repository with the license changed in a non-compatible manner. As part of this, the attribution required by the license was also omitted. The details of this can be found further down in this post.

Like the majority of the Linux kernel source code, the datapath portion of Cilium that runs as part of the Linux kernel is released under the GPL 2.0 license. The GPL license does not permit a license change to the Apache License without consent of the original authors.

This prompted us to contact the authors of the derivative work. As a result, an initial attempt was made to rewrite some sections of the code. After inspection, we concluded that the work is still a derivative of our original source code.

However, in order to resolve the situation as simply as possible, we offered to dual-license the respective code under the Apache license with the condition that attribution to the original authors is added. This resulted in the following pull request being proposed to the Calico repository to add the attribution. The pull request is currently waiting to be merged.

From our perspective, this would resolve all of our concerns. We obviously also accept any other resolution as long as it conforms to the respective open source licenses.

We are waiting for a reaction by the maintainers of the Calico project.

Update 2019-06-25: Some of the eBPF related code has now been removed from the Calico repository via this PR.

Back in March we have asked our users to provide feedback via our first ever user survey. Many of you have responded and the results are in!

The survey was announced on our Slack channel and on Twitter. Participation was anonymous and did not require to leave behind contact information. Most questions had a set of predefined answers plus a field to add additional answers. All questions were optional.

We are excited to announce the Cilium 1.5 release. Cilium 1.5 is the first release where we primarily focused on scalability with respect to number of nodes, pods and services. Our goal was to scale to 5k nodes, 20k pods and 10k services. We went well past that goal with the 1.5 release and are now officially supporting 5k nodes, 100k pods and 20k services. Along the way, we learned a lot, some expected, some unexpected, this blog post will dive into what we learned and how we improved.

Besides scalability, several significant features made its way into the release including: BPF templating, rolling updates for transparent encryption keys, transparent encryption for direct-routing, a new improved BPF based service load-balancer with improved fairness, BPF based masquerading/SNAT support, Istio 1.1.3 integration, policy calculation optimizations as well as several new Prometheus metrics to assist in operations and monitoring. For the full list of changes, see the 1.5 Release Notes.

This is a deep dive into ClusterMesh, Cilium's multi-cluster implementation. In a nutshell, ClusterMesh provides:

• Pod IP routing across multiple Kubernetes clusters at native performance via tunneling or direct-routing without requiring any gateways or proxies.

• Transparent service discovery with standard Kubernetes services and coredns/kube-dns.

• Network policy enforcement spanning multiple clusters. Policies can be specified as Kubernetes NetworkPolicy resource or the extended CiliumNetworkPolicy CRD.

• Transparent encryption for all communication between nodes in the local cluster as well as across cluster boundaries.

We are excited to announce the Cilium 1.4 release. The release introduces several new features as well as optimization and scalability work. The highlights include the addition of global services to provide Kubernetes service routing across multiple clusters, DNS request/response aware authorization and visibility, transparent encryption (beta), IPVLAN support for better performance and latency (beta), integration with Flannel, GKE on COS support, AWS metadata based policy enforcement (alpha) as well as significant efforts into optimizing memory and CPU usage.

As we all enjoy a wonderful week at KubeCon 2018 US, we want to provide a preview into the upcoming Cilium 1.4 release. We are days away from 1.4.0-rc1 which will allow for community testing of a lot new exciting functionality. Some of the highlights:

• Multi-Cluster service routing using standard Kubernetes services.
• DNS Authorization with DNS request/response aware security policy enforcement to restrict the DNS names a pod can lookup as well as limit the egress connectivity to the IPs returned in the DNS response of that particular pod.
• Transparent encryption and authentication for all service to service communication using X.509 certificates.

As always, we love hearing from you, so stop by our KubeCon booth and chat with us and other Cilium users.

First of all, huge shout-out to Alexis Ducastel for putting together a great CNI benchmark comparison. To be honest, there was definitely a moment of panic when we saw the article pop up. Did we just miss a major performance regression?

This blog post documents the investigation we have done so far of what looked like a performance regression of HTTP/FTP traffic over pure TCP.

Alexis was super quick to share the scripts that he used to collect the benchmarks numbers. This not only allowed for a quick verification but also allows us to integrate this into our CI tests and run it alongside of the existing benchmarks for better coverage.