KubeCon North America 2020 Wrap-up

KubeCon + CloudNativeCon North America 2020 Virtual (say that five times fast) was held online, November 18th to 20th. It was the second virtual KubeCon, due to the Coronavirus pandemic. I had a lot of fun attending and saw a lot of great talks.

If you weren’t able to participate in real-time, the CNCF will post the talks soon to its YouTube channel.

Day One - Wednesday

Keynotes

There were many tributes to Dan Kohn throughout the keynotes. Dan was the Executive Director of the CNCF (Cloud Native Computing Foundation), and he passed away on November 1st. It was clear from listening to people that Dan touched many lives and had a huge impact on the CNCF and Kubernetes. I found the tributes to Dan very moving. If you would like to leave a message celebrating Dan’s life, there’s a GitHub repo with memorials to Dan.

Priyanka Sharma was up first, talking about the importance of diversity and inclusion. I was happy to see this topic raised right at the beginning of the conference. I think it set a great tone. Priyanka talked about the importance of inclusive language and mentioned that people in the Kubernetes community are working to remove non-inclusive terms like master and slave from the code base. It’s an effort that I applaud. (There’s a group called WG Naming, and Celeste Horgan spoke about it later in the conference.)

Unfortunately, there was a lot of streaming pain during the keynotes, but it improved later in the day. The stream kept freezing up. I thought it was just my internet at first, but other people reported the same problems. I was a bit surprised as this hadn’t been an issue at the last KubeCon, but it was one of the few speed bumps in a very smooth conference.

Cheryl Hung spoke about Kubernetes certifications. The CNCF has launched the CKS exam (Certified Kubernetes Security Specialist), which is now the third Kubernetes certification. I’m happy to see the CNCF focusing on security even more. Cheryl did a demo of the exam, which was great to see.

Next, Alena Prokharchyk spoke about Kubernetes at Apple. Alena was at Apple through the company’s migration from Mesos to Kubernetes. Much of their focus has been on the internal developers’ experience, which is a great move. This talk was a big victim of the streaming problems, unfortunately, and I had some trouble following the flow. But it was great to see someone from Apple on the stage, as the company has been hiring a lot of top Kubernetes talent. It’s made some of us very curious.

Constance Caramanolis was one of the conference’s co-chairs, and she gave updates next on CNCF projects. There were a lot of cool metrics, like these for Falco:

And wrapping up the morning’s keynotes was the other co-chair, Stephen Augustus. Stephen spoke about the community and how people have been very thoughtful during the pandemic. He also mentioned that the CNCF had made an unequivocal rejection of racism and that it meant a lot to him. Stephen has done a lot for the Kubernetes community. He’s someone who is in the middle of a lot of significant efforts and brings people together. He was the first person I invited to be a guest on my Kubernetes podcast, and that’s why. Stephen and Constance did a fantastic job chairing the conference.

Joe Thompson - Stop Writing Operators

The first talk I caught after the keynotes was Joe Thompson from HashiCorp talking about why you shouldn’t be writing Kubernetes operators.

I love talks like this that discuss why we shouldn’t fall into using a tool or a pattern. Joe mentioned that there are better ways to manage your application than reaching for an operator if there’s no state in the app. He suggested building the management logic into the app itself, if possible, or into the Helm chart. Joe also recommended against writing operators for apps that aren’t stable from a development standpoint. If the APIs are changing a lot, that’s not a great time to code an operator.

If you are working with operators, this would be a great talk to watch. Or if you’d like some chuckles.

Leigh Capili - Strategies for Multi-Cluster Routing

I was a little intimidated about attending the next talk from Leigh Capili.

Most of my routing experience has been on the host level, and I don’t know much about overlay networks. But Leigh did a great job explaining things, and I managed to follow quite a lot of it. He started by talking about why people do multi-cluster (some for technical reasons, and some for business reasons). And then he got into some of the networking details, like routing protocols.

Leigh does fantastic demos, and this talk was not an exception. You can walk through the demo yourself using this GitHub repo. It makes me happy when people share their demo code.

Qian Ding and Cong Chen - An SOL-Driven Approach to Enhance Kubernetes Cluster Reliability

This was a great talk on SLOs (Service Level Objectives) from a team providing Kubernetes services to other engineers in an organization. I’m used to thinking about SLOs in terms of apps the engineers have written themselves and hadn’t considered how teams might set SLOs for Kubernetes itself. The speakers covered SLO design aspects and dug into some of the math around alerting.

One of the signs that this was a great talk was the reactions from Liz Fong-Jones, who’s an expert in SRE.

It’s worth watching if you’d like to learn more about SLOs, and how to use them to make Kubernetes more reliable.

Day Two - Thursday

Keynotes

First up was Constance Caramanolis talking about OpenTelemetry, which launched last year to replace two competing standards, OpenCensus and OpenTracing. It was great to see folks from the Observability community come together around a single standard, which is now getting support from vendors in the tracing space. Constance talked about the concerns people have with vendor lock-in and how OpenTelemetry addresses them by supporting multiple standards. She also talked through the migration path, which can start with just the OpenTelemetry collector.

Next, Jonathan Beri spoke about the need to support more protocols beyond HTTP. Jonathan mentioned gaming and IoT as use cases, and he went into K8s Gateway.

Jonathan has been gathering feedback from the community about supporting other protocols, and he has a Google Doc that folks can add their feedback to at bitly.com/beyondhttp.

Wojciech Tyczyński gave the next keynote about running Kubernetes clusters with 15k nodes at Google. There are a lot of considerations when it comes to scaling.

He mentioned specific changes made to apimachinery and etcd, and said that the changes also benefit people running smaller clusters.

It took me a bit to recognize the next presenter, David Sudia, who spoke about a Kubernetes platform his team built. David previously gave a talk I enjoyed called “If You Can Wait 6 Months, You Should,” at the Deserted Island DevOps conference. As you might tell from that title, David is very pragmatic, and he mentioned some of the concerns they had about making a platform for their internal engineers to use.

His team took a real product focus, which is something I appreciate in infrastructure projects.

If you are supporting internal developers using Kubernetes clusters, this talk would be great to watch for some perspective.

Joaquim Rocha and Henning Jacobs - A Walk Through the Kubernetes UI Landscape

I started the breakout sessions watching this very interesting talk from Joaquim Rocha and Henning Jacobs about some of the open source Kubernetes UIs that are available. This was an excellent overview from two engineers who have tried many of the Kubernetes UIs and have helped write some as well. Here are some of the factors they looked at.

Some of the considerations were very interesting. For example, they mentioned that in shops doing GitOps, you might want a read-only UI, so people don’t apply changes to the cluster outside of the standard workflow.

After walking through the pros and cons of the different projects, they presented this handy comparison chart.

Stefan Prodan - Progressive Delivery Techniques with Flagger

Flagger is a very interesting tool that I hadn’t been aware of. In this talk, Stefan Prodan explained why Weaveworks built a tool to support Progressive Delivery. When I think about the practices that make up Progressive Delivery, like canaries and feature flags, I think of deployment safety. And deployment safety is critical for teams to reduce stress around deployments and incidents.

Stefan discussed some of the challenges with Continuous Delivery:

He also talked about Flagger’s architecture and gave a demo of it.

I got a heads up on Twitter that there’s also a TGIK episode about Flagger if you’d like to see more (thanks, Timmy Carr).

Jian Cheung and Joseph Kim - 10 More Weird Ways to Blow Up Your Kubernetes Cluster

As I suspected, this talk was a sequel to the excellent KubeCon 2019 talk by Melanie Cebula and Bruce Sherrod. The Airbnb Engineering team has run into some weird and fascinating issues running Kubernetes, and I appreciate that they continue to share the things they’ve learned with the community.

Day Three - Friday

The final day’s keynotes started with Stephen Augustus talking about the 2020 Kubernetes release cadence changes. The pandemic has taken a pretty big toll on people in many different ways, and as a response, the releases were scaled back this year from four to three. That was a smart and thoughtful decision. Now the team is evaluating what to do in 2021.

It will be interesting to see how this plays out. I think it’s going to be a while yet before we’re out of the woods with Covid, and as Stephen mentioned, some people already wanted fewer releases (and LTS).

Next up was a panel of security experts answering questions about Kubernetes security, made up of Ian Coldwater, Duffie Cooley, Brad Geesaman, and Rory McCune. When I looked at the KubeCon schedule, this is a talk that jumped out at me immediately. There were a lot of great thoughts shared.

This is a talk I think everyone using Kubernetes should watch.

After the panel, Liz Rice gave some updates from the TOC.

Liz made many great points about how we need easier workflows for developers, and we need to do a better job explaining solutions that fit people’s use cases.

Chris Aniszczyk wrapped up the keynotes with the CNCF Maintainer and Ambassador awards.

It was great to see some folks I know on the list. They are all well-deserving of the recognition.

Cornelia Davis - GitOps Is Likely More Than You Think It Is

This talk was a good overview of GitOps, explicitly applied to Kubernetes. I dealt with application deployments and configuration for much of my career, and watching how things are evolving is very interesting.

Cornelia talked about the benefits of pulling config, as opposed to pushing it, and how to do Continuous Delivery with Kubernetes. She focused on a few tools, like Kustomize and Flagger.

I pay attention to how many times I hear a tool or pattern mentioned at a conference. This was the second time that Flagger came up, making me think I should look into it more.

Jeremy Rickard - Using Open Policy Agent to Meet Evolving Policy Requirements

The last talk I attended was from Jeremy Rickard, about Open Policy Agent. Jeremy talked about how his team used OPA to help enforce requirements in a regulated environment. If your systems are subject to PCI DSS audits or other regulations, Jeremy’s talk would be great to watch.

Jeremy explained OPA and why they chose to use it, and gave several examples of policies his team put into place.

His team even made rules that work with mutating webhooks to enforce policies on the fly.

This talk was a great look at using OPA with specific code examples that were very useful.

And that was a wrap for KubeCon 2020. I had a great time attending. If you weren’t able to attend, keep an eye out on the CNCF’s YouTube channel for videos of the talks.

I’m looking forward to a physical KubeCon where I can see my friends in the Kubernetes community again, but in the meantime, I really appreciate these virtual events. Thanks to the CNCF, the conference chairs, the speakers, and everyone else involved in putting on the show. See you in May for KubeCon Europe.