6. Upcoming Serverless Features in Kubernetes – Serverless Architectures with Kubernetes

6. Upcoming Serverless Features in Kubernetes

Learning Objectives

By the end of this chapter, you will be able to:

  • Utilize the concepts and components of Knative to deploy applications
  • Set up Knative on a GKE cluster
  • Deploy applications on Knative and configure autoscaling
  • Deploy applications on Google Cloud Run
  • Set up Virtual Kubelet on Azure
  • Deploy applications with Virtual Kubelet

This chapter covers Knative, Google Cloud Run, and Virtual Kubelet, which offers the advantages of serverless on top of a Kubernetes cluster.

Introduction to Serverless with Kubernetes

In the previous chapter, we extensively studied the various setup options and platforms used in Kubernetes. We also covered the autoscaling feature of Kubernetes and implemented it in an application deployed on a cluster.

Kubernetes and serverless are two of the trending topics in the IT industry, but these two topics are often discussed independently of each other. Kubernetes is a platform for managing containerized applications, and serverless is an execution model that abstracts away the infrastructure so software developers can focus on their application logic. However, a combination of these two concepts will achieve the same goal of making the software developer's life much easier.

A few platforms have emerged recently that bring serverless features to containers by abstracting away the complexities of managing containers and any underlying infrastructure. These platforms run serverless workloads on Kubernetes clusters and provide many benefits, including autoscaling, scale to zero, per-usage billing, event-driven capabilities, integrated monitoring, and integrated logging features.

In this chapter, we will be discussing three technologies that offer the benefits of serverless on top of a Kubernetes cluster:

  • Knative
  • Google Cloud Run
  • Virtual Kubelet

Introduction to Knative

Knative is an open source project started by Google with contributions from over 50 other companies, including Pivotal, Red Hat, IBM, and SAP. Knative extends Kubernetes by introducing a set of components to build and run serverless applications on top of it. This framework is great for application developers who are already using Kubernetes. Knative provides tools for them to focus on their code without worrying about the underlying architecture of Kubernetes. It introduces features such as automated container builds, autoscaling, scale to zero, and an eventing framework, which allows developers to get the benefits of serverless on top of Kubernetes.

The Knative framework is described as a "Kubernetes-based platform to deploy and manage modern serverless workloads" on the Knative website. The framework helps to bridge the gap between containerized applications and serverless applications by introducing serverless features such as autoscaling and scale to zero to the Kubernetes platform.

Knative consists of three main components:

  • Build
  • Serving
  • Eventing

    Note

    The Build component has been deprecated in favor of Tekton Pipelines in the latest version of Knative. The final release of the Knative Build component is available in version 0.7.

Build is the process of building the container images from the source code and running them on a Kubernetes cluster. The Knative Serving component allows the deployment of serverless applications and functions. This enables serving traffic to containers and autoscaling based on the number of requests. The serving component is also responsible for taking snapshots of the code and configurations whenever a change is made to them. The Knative Eventing component helps us to build event-driven applications. This component allows the applications to produce events for and consume events from event streams.

The following diagram illustrates a Knative framework with its dependencies and the stakeholders of each component:

Figure 6.1: Knative dependencies and stakeholders

The bottom layer represents the Kubernetes framework, which is used as the container orchestration layer by the Knative framework. Kubernetes can be deployed on any infrastructure, such as Google Cloud Platform or an on-premises system. Next, we have the Istio service mesh layer, which manages network routing within the cluster. This layer provides many benefits, including traffic management, observability, and security. At the top layer, Knative runs on top of a Kubernetes cluster with Istio. In the Knative layer, at one end we can see contributors who contribute code to the Knative framework through the GitHub project, and at the other end we can see the application developers who build and deploy applications on top of the Knative framework.

Note

For more information on Istio, please refer to https://istio.io/.

Now that we have this understanding of Knative, let's look at how to install Knative on a Kubernetes cluster in the following section.

Getting Started with Knative on GKE

In this section, we will take you through the process of installing Knative on a Kubernetes cluster. We will be using Google Kubernetes Engine (GKE) to set up a Kubernetes cluster. GKE is the managed Kubernetes cluster service in the Google cloud. It allows us to run Kubernetes clusters without the burden of installing, managing and operating our own clusters.

We need to have the following prerequisites installed and configured to continue with this section:

  • A Google Cloud account
  • The gcloud CLI
  • The kubectl CLI (v1.10 or newer)

First, we need to set a few environment variables that we will be using with the gcloud CLI. You should update <your-gcp-project-name> with the name of your GCP project. We will be using us-central1-a as the GCP zone. Execute the following commands in your terminal window to set the required environment variables:

$ export GCP_PROJECT=<your-gcp-project-name>

$ export GCP_ZONE=us-central1-a

$ export GKE_CLUSTER=knative-cluster

The output should be as follows:

Figure 6.2: Setting environment variables

Set our GCP project as the default project to be used by the gcloud CLI commands:

$ gcloud config set core/project $GCP_PROJECT

The output should be as follows:

Figure 6.3: Setting the default GCP project

Now we can create the GKE cluster using the gcloud command. Knative requires a Kubernetes cluster with version 1.11 or newer. We will be using the Istio plugin provided by GKE for this cluster. The following is the recommended configuration for a Kubernetes cluster to run Knative components:

  • Kubernetes version 1.11 or newer
  • Kubernetes nodes with four vCPUs (n1-standard-4)
  • Node autoscaling enabled for up to 10 nodes
  • API scopes for cloud-platform

Execute the following command to create a GKE cluster compatible with these requirements:

     $ gcloud beta container clusters create $GKE_CLUSTER \

    --zone=$GCP_ZONE \

    --machine-type=n1-standard-4 \

    --cluster-version=latest \

    --addons=HorizontalPodAutoscaling,HttpLoadBalancing,Istio \

    --enable-stackdriver-kubernetes \

    --enable-ip-alias \

    --enable-autoscaling --min-nodes=1 --max-nodes=10 \

    --enable-autorepair \

    --scopes cloud-platform

The output should be as follows:

Figure 6.4: Creating a GKE cluster

It may take a few minutes to set up the Kubernetes cluster. Once the cluster is ready, we will use the command gcloud container clusters get-credentials to fetch the credentials of the new cluster and configure the kubectl CLI as you can see in the following code snippet:

$ gcloud container clusters get-credentials $GKE_CLUSTER --zone $GCP_ZONE --project $GCP_PROJECT

The output should be as follows:

Figure 6.5: Fetching credentials for the GKE cluster

Now you have successfully created the GKE cluster with Istio and configured kubectl to access the newly created cluster. We can now proceed with the next step of installing Knative. We will be installing Knative version 0.8, which is the latest available version at the time of writing this book.

We will use the kubectl CLI to apply the Knative components to the Kubernetes cluster. First, run the kubectl apply command with the -l knative.dev/crd-install=true flag to prevent race conditions during the installation process:

$ kubectl apply --selector knative.dev/crd-install=true \

   -f https://github.com/knative/serving/releases/download/v0.8.0/serving.yaml \

   -f https://github.com/knative/eventing/releases/download/v0.8.0/release.yaml \

   -f https://github.com/knative/serving/releases/download/v0.8.0/monitoring.yaml

Next, run the command again without the -l knative.dev/crd-install=true flag to complete the installation:

$ kubectl apply -f https://github.com/knative/serving/releases/download/v0.8.0/serving.yaml \

   -f https://github.com/knative/eventing/releases/download/v0.8.0/release.yaml \

   -f https://github.com/knative/serving/releases/download/v0.8.0/monitoring.yaml

Once the command is completed, execute the following commands to check the status of the installation. Make sure that all pods have a status of Running:

$ kubectl get pods --namespace knative-serving

$ kubectl get pods --namespace knative-eventing

$ kubectl get pods --namespace knative-monitoring

The output should be as follows:

Figure 6.6: Verifying Knative installation

At this stage, you have set up a Kubernetes cluster on GKE and installed Knative. Now we are ready to deploy our first application on Knative.

Exercise 16: Deploying a Sample Application on Knative

In the previous section, we successfully deployed Knative on top of Kubernetes and Istio. In this exercise, we will deploy our first application on the Knative framework. For this deployment, we are going to use a sample web application written with Node.js. A Docker image of this application is available in Google Container Registry at gcr.io/knative-samples/helloworld-nodejs. These steps can be adapted to deploy our own Docker image on Docker Hub or any other container registry.

This sample "hello world" application will read an environment variable named TARGET and print Hello <VALUE_OF_TARGET>! as the output. It will print NOT SPECIFIED as the output if no value is defined for the TARGET environment variable.

Let's start by creating the service definition file for our application. This file defines application-related information including the application name and the application Docker image:

Note

Knative service objects and Kubernetes Service objects are two different types.

  1. Create a file named hello-world.yaml with the following content. This Knative service object defines values such as the namespace to deploy this service in, the Docker image to use for the container, and any environment variables:

              apiVersion: serving.knative.dev/v1alpha1

    kind: Service

    metadata:

      name: helloworld-nodejs

      namespace: default

    spec:

      runLatest:

        configuration:

          revisionTemplate:

            spec:

              container:

                image: gcr.io/knative-samples/helloworld-nodejs

                env:

                  - name: TARGET

                    value: "Knative NodeJS App"

  2. Once the hello-world.yaml file is ready, we can deploy our application with the kubectl apply command:

    $ kubectl apply -f hello-world.yaml

    The output should be as follows:

    Figure 6.7: Deploying the helloworld-nodejs application
  3. The previous command will create multiple objects, including the Knative service, configuration, revision, route, and Kubernetes Deployment. We can verify the application by listing the newly created objects as in the following commands:

    $ kubectl get ksvc

    $ kubectl get configuration

    $ kubectl get revision

    $ kubectl get route

    $ kubectl get deployments

    The output should be as follows:

    Figure 6.8: Verifying helloworld-nodejs application deployment
  4. Once our application is deployed successfully, we can invoke this application using an HTTP request. For this, we need to identify the external IP address of the Kubernetes cluster. Execute the following command to export the value of EXTERNAL-IP into an environment variable named EXTERNAL_IP:

    $ export EXTERNAL_IP=$(kubectl get svc istio-ingressgateway --namespace istio-system --output 'jsonpath={.status.loadBalancer.ingress[0].ip}')

    The output should be as follows:

    Figure 6.9: Exporting the external IP of the istio-ingressgateway service

    Next, we need to find the host URL of the helloworld-nodejs application. Execute the following command and take note of the value of the URL column. This URL takes the form http://<application-name>.<namespace>.example.com:

    $ kubectl get route helloworld-nodejs

    The output should be as follows:

    Figure 6.10: Listing the helloworld-nodejs route
  5. Now we can invoke our application using the EXTERNAL_IP and URL values that we noted in the earlier steps. Let's make a curl request with the following command:

    $ curl -H "Host: helloworld-nodejs.default.example.com" http://${EXTERNAL_IP}

    The output should be as follows:

Figure 6.11: Invoking the helloworld-nodejs application

You should receive the expected output as Hello Knative NodeJS App!. This indicates that we have successfully deployed and invoked our first application on the Knative platform.

Knative Serving Component

In the previous section, we deployed our first Knative application using a YAML file of the service type. When deploying the service, it created multiple other objects, including configuration, revision, and route objects. In this section, let's discuss each of these objects:

There are four resource types in the Knative Serving component:

  • Configuration: Defines the desired state of the application
  • Revision: Read-only snapshots that track the changes in configurations
  • Route: Provides traffic routing to revisions
  • Service: Top-level container for routes and configurations

The following diagram illustrates the relationship between each of these components:

Figure 6.12: Relationship between Knative services, routes, configurations, and revisions

The configuration is used to define the desired state of the application. This will define the container image used for the application and any other configuration parameters that are required. A new Revision will be created each time a Configuration is updated. Revision refers to a snapshot of the code and the Configuration. This is used to record the history of Configuration changes. A Route is used to define the traffic routing policy of the application and provides an HTTP endpoint for the application. By default, the Route will send traffic to the latest Revision created by the Configuration. The Route can also be configured for more advanced scenarios, including sending traffic to a specific Revision or splitting traffic to different revisions based on defined percentages. Service objects are used to manage the whole life cycle of the application. While deploying a new application, it is required to create Configuration and Route objects manually, but the Service can be used to simplify this by creating and managing Configuration and Route objects automatically.

In the following section, we will be using canary deployment to deploy applications with Knative. Let's first understand what exactly canary deployment is.

Canary Deployment

Canary deployment is a deployment strategy used when rolling out a new version of code to a production environment. This is a fail-safe process of deploying a new version of code into a production environment and switching a small percentage of traffic to the new version. This way, the development and deployment teams can verify the new version of the code with minimal impact on production traffic. Once the verifications are done, all traffic will be switched to the new version. In addition to canary deployments, there are several other deployment types, such as big bang deployments, rolling deployments, and blue-green deployments.

In the helloworld-nodejs application that we deployed in Exercise 16, Deploying a Sample App on Knative, we used the Service object with the spec.runLatest field, which directs all traffic to the latest available revision. In the following exercise, we will be using separate configuration and route objects instead of the service object.

Note

For more information on canary deployment technique, refer to https://dev.to/mostlyjason/intro-to-deployment-strategies-blue-green-canary-and-more-3a3.

Exercise 17: Canary Deployment with Knative

In this exercise, we will be implementing a canary deployment strategy to deploy applications with Knative. First, we will deploy an initial version (version 1) of an application and route 100% traffic to that version. Next, we will create version 2 of the application and route 50% of traffic to version 1 and the remaining 50% to version 2. Finally, we will update the routes to send 100% of traffic to version 2.

The following steps will help you complete the exercise:

  1. First, start by creating the initial version (v1) of the application. Create a file named canary-deployment.yaml with the following content. This application uses the same Docker image (gcr.io/knative-samples/helloworld-nodejs) that we used previously and sets the TARGET environment variable as This is the first version - v1:

    apiVersion: serving.knative.dev/v1alpha1

    kind: Configuration

    metadata:

      name: canary-deployment

      namespace: default

    spec:

      template:

        spec:

          containers:

            - image: gcr.io/knative-samples/helloworld-nodejs

              env:

                - name: TARGET

                  value: "This is the first version - v1"

  2. Deploy the first version of the application with the kubectl apply command using the YAML file created in the previous step:

    $ kubectl apply -f canary-deployment.yaml

    The output should be as follows:

    Figure 6.13: Creating canary-deployment
  3. Let's get the revision name created by this configuration as we need this value in the next step. Execute the kubectl get configurations command and retrieve the value of the latestCreatedRevisionName field:

    $ kubectl get configurations canary-deployment -o=jsonpath='{.status.latestCreatedRevisionName}'

    The output should be as follows:

    Figure 6.14: Getting the latest revision of the canary-deployment configuration

    For me, the value returned from the preceding command is canary-deployment-xgvl8. Note that your value will be different.

  4. The next step is to create the route object. Let's create a file named canary-deployment-route.yaml with the following content (please remember to replace canary-deployment-xgvl8 with the revision name that you noted in the previous step). Under the spec.traffic section, you can see that 100% of traffic is routed to the revision that we created previously:

    apiVersion: serving.knative.dev/v1alpha1

    kind: Route

    metadata:

      name: canary-deployment

      namespace: default

    spec:

      traffic:

        - revisionName: canary-deployment-xgvl8

          percent: 100

  5. Create the route object with the kubectl apply command:

    $ kubectl apply -f canary-deployment-route.yaml

    The output should be as follows:

    Figure 6.15: Creating the canary-deployment route
  6. Make a request to the application and observe the expected output of Hello This is the first version - v1!:

    $ curl -H "Host: canary-deployment.default.example.com" "http://${EXTERNAL_IP}"

    The output should be as follows:

    Figure 6.16: Invoking canary-deployment
  7. Once the application is successfully invoked, we can deploy version 2 of the application. Update canary-deployment.yaml with the following content. In version 2 of the application, we only need to update the value of the TARGET environment variable from This is the first version - v1 to This is the second version - v2:

    apiVersion: serving.knative.dev/v1alpha1

    kind: Configuration

    metadata:

      name: canary-deployment

      namespace: default

    spec:

      template:

        spec:

          containers:

            - image: gcr.io/knative-samples/helloworld-nodejs

              env:

                - name: TARGET

                  value: "This is the second version - v2"

  8. Apply the updated configuration with kubectl apply:

    $ kubectl apply -f canary-deployment.yaml

    The output should be as follows:

    Figure 6.17: Updating canary-deployment to version 2
  9. Now we can check the revisions created, while updating the configuration, using the kubectl get revisions command:

    $ kubectl get revisions

    The output should be as follows:

    Figure 6.18: Getting the revisions of canary-deployment
  10. Let's get the latest revision created by the canary-deployment configuration:

    $ kubectl get configurations canary-deployment -o=jsonpath='{.status.latestCreatedRevisionName}'

    The output should be as follows:

    Figure 6.19: Getting the latest revision of the canary-deployment configuration
  11. Now it's time to send some traffic to our new version of the application. Update the spec.traffic section of canary-deployment-route.yaml to send 50% of the traffic to the old revision and 50% to the new revision:

    apiVersion: serving.knative.dev/v1alpha1

    kind: Route

    metadata:

      name: canary-deployment

      namespace: default

    spec:

      traffic:

        - revisionName: canary-deployment-xgvl8

          percent: 50

        - revisionName: canary-deployment-8pp4s

          percent: 50

  12. Apply changes to the route using the following command:

    $ kubectl apply -f canary-deployment-route.yaml

    The output should be as follows:

    Figure 6.20: Updating the canary-deployment route
  13. Now we can invoke the application multiple times to observe how traffic splits between two revisions:

    $ curl -H "Host: canary-deployment.default.example.com" "http://${EXTERNAL_IP}"

  14. Once we verify version 2 of the application successfully, we can update canary-deployment-route.yaml to route 100% of the traffic to the latest revision:

    apiVersion: serving.knative.dev/v1alpha1

    kind: Route

    metadata:

      name: canary-deployment

      namespace: default

    spec:

      traffic:

        - revisionName: canary-deployment-xgvl8

          percent: 0

        - revisionName: canary-deployment-8pp4s

          percent: 100

  15. Apply the changes to the route using the following command:

    $ kubectl apply -f canary-deployment-route.yaml

    The output should be as follows:

    Figure 6.21: Updating the canary-deployment route
  16. Now invoke the application multiple times to verify that all traffic goes to version 2 of the application:

    $ curl -H "Host: blue-green-deployment.default.example.com" "http://${EXTERNAL_IP}"

In this exercise, we have successfully used configuration and route objects to perform a canary deployment with Knative.

Knative Monitoring

Knative comes with Grafana pre-installed, which is an open source metric analytics and visualization tool. The Grafana pod is available in the knative-monitoring namespace and can be listed with the following command:

$ kubectl get pods -l app=grafana -n knative-monitoring

The output should be as follows:

Figure 6.22: Listing the Grafana pod

We can expose the Grafana UI with the kubectl port-forward command, which will forward local port 3000 to the port 3000 of the Grafana pod. Open a new terminal and execute the following command:

$ kubectl port-forward $(kubectl get pod -n knative-monitoring -l app=grafana -o jsonpath='{.items[0].metadata.name}') -n knative-monitoring 3000:3000

The output should be as follows:

Figure 6.23: Port forwarding to the Grafana pod

Now we can navigate the Grafana UI from our web browser on http://127.0.0.1:3000.

The output should be as follows:

Figure 6.24: The Grafana UI

Knative's Grafana dashboard comes with multiple dashboards, including the following:

Figure 6.25: Dashboards

Knative Autoscaler

Knative has a built-in autoscaling feature that automatically scales the application pods based on the number of HTTP requests it receives. This will increase the pod count when there is increased demand and decrease the pod count when the demand decreases. The pod count will scale to zero when pods are idle and there are no incoming requests.

Knative uses two components, the autoscaler, and the activator, to achieve the previously mentioned functionality. These components are deployed as pods in the knative-serving namespace, as you can see in the following snippet:

NAME                          READY   STATUS    RESTARTS   AGE

activator-7c8b59d78-9kgk5     2/2     Running   0          15h

autoscaler-666c9bfcc6-vwrj6   2/2     Running   0          15h

controller-799cd5c6dc-p47qn   1/1     Running   0          15h

webhook-5b66fdf6b9-cbllh      1/1     Running   0          15h

The activator component is responsible for collecting information about the number of concurrent requests to a revision and reporting these values to the autoscaler. The autoscaler component will increase or decrease the number of pods based on the metrics reported by the activator. By default, the autoscaler will try to maintain 100 concurrent requests per pod by scaling pods up or down. All Knative autoscaler-related configurations are stored in a configuration map named config-autoscaler in the knative-serving namespace. Knative can also be configured to use the Horizontal Pod Autoscaler (HPA), which is provided by Kubernetes. HPA will autoscale pods based on CPU usage.

Exercise 18: Autoscaling with Knative

In this exercise, we will perform Knative pod autoscaling by deploying a sample application:

  1. Create an autoscale-app.yaml service definition file with the following content. This file defines a service named autoscale-app, which will use the gcr.io/knative-samples/autoscale-go:0.1 sample Docker image. autoscaling.knative.dev/target is used to configure the target number of concurrent requests per pod:

    apiVersion: serving.knative.dev/v1alpha1

    kind: Service

    metadata:

      name: autoscale-app

    spec:

      runLatest:

        configuration:

          revisionTemplate:

            metadata:

              annotations:

                autoscaling.knative.dev/target: "10"

            spec:

              container:

                image: "gcr.io/knative-samples/autoscale-go:0.1"

  2. Apply the service definition with the kubectl apply command:

    $ kubectl apply -f autoscale-app.yaml

    The output should be as follows:

    Figure 6.26: Creating autoscale-app
  3. Once the application is ready, we can generate a load to the autoscale-app application to observe the autoscaling. For this, we will use a load generator named hey. Download the hey binary using the following curl command.

    $ curl -Lo hey https://storage.googleapis.com/hey-release/hey_linux_amd64

    The output should be as follows:

    Figure 6.27: Installing hey
  4. Add execution permission to the hey binary and move it into the /usr/local/bin/ path:

    $ chmod +x hey

    $ sudo mv hey /usr/local/bin/

    The output should be as follows:

    Figure 6.28: Moving hey to /usr/local/bin
  5. Now we are ready to generate a load with the hey tool. The hey tool supports multiple options when generating a load. For this scenario, we will use a load with a concurrency of 50 (with the -c flag) for a duration of 60 seconds (with the -z flag):

    $ hey -z 60s -c 50 \

       -host "autoscale-app.default.example.com" \

       "http://${EXTERNAL_IP?}?sleep=1000"

  6. In a separate terminal, watch for the number of pods created during the load:

    $ kubectl get pods --watch

    You will see output similar to the following:

         NAME                                             READY   STATUS    RESTARTS   AGE

    autoscale-app-7jt29-deployment-9c9c4b474-4ttl2   3/3     Running   0          58s

    autoscale-app-7jt29-deployment-9c9c4b474-6pmjs   3/3     Running   0          60s

    autoscale-app-7jt29-deployment-9c9c4b474-7j52p   3/3     Running   0          63s

    autoscale-app-7jt29-deployment-9c9c4b474-dvcs6   3/3     Running   0          56s

    autoscale-app-7jt29-deployment-9c9c4b474-hmkzf   3/3     Running   0          62s

  7. Open the Knative Serving - Scaling Debugging dashboard from Grafana to observe how autoscaling increased the pod count during the load and decreased the pod count back to zero once the load stopped, as you can see in the following screenshots:
Figure 6.29: Revision pod count metrics
Figure 6.30: Observed concurrency metrics

We have successfully configured Knative's autoscaler and observed autoscaling with the Grafana dashboard.

Google Cloud Run

In the previous sections, we discussed Knative. We learned how to install Istio and Knative on top of a Kubernetes cluster and how to run Docker images with Knative. But the advantages of the Knative platform come with the operational overhead of managing the underlying Kubernetes cluster with Istio. GKE, which is the managed Kubernetes service from Google Cloud, will help us manage the Kubernetes master components, but still, we have to manage all the Kubernetes nodes ourselves.

In order to abstract away all the infrastructure management tasks from the developer, Google introduced a new service named Cloud Run. This is a fully managed platform, built on the Knative project, to run stateless HTTP-driven containers. Cloud Run offers the same set of features as Knative, including autoscaling, scale to zero, versioning, and events. Cloud Run was introduced in the Google Cloud Next '19 conference as the newest member of Google Cloud's serverless compute stack. At the time of writing this book, the Cloud Run service is still in beta and only available in a limited number of regions.

Let's now perform an exercise to deploy containers on Google Cloud Run.

Exercise 19: Deploying Containers on Google Cloud Run

In this exercise, we will be deploying a pre-built Docker image on the Google Cloud Run platform.

The following steps will help you complete the exercise:

  1. Navigate to your GCP console from your browser and select Cloud Run from the menu (in the Compute category) as shown in the following figure:
    Figure 6.31: GCP menu for Cloud Run
  2. Click on the CREATE SERVICE button to create a new service.
  3. Fill the create service form with the following values:

    Container Image URL: gcr.io/knative-samples/helloworld-nodejs

    Deployment platform: Cloud Run (fully managed)

    Location: Select any region you prefer from the options

    Service name: hello-world

    Authentication: Allow unauthenticated invocations

    Figure 6.32: Cloud Run create service form
  4. Click on the CREATE button.
  5. Now we will be redirected to the deployed service page, which includes details about the newly deployed hello-world service. We can see that a revision has been created called hello-world-00001, as shown in the following figure:
    Figure 6.33: Service details page
  6. Click on the URL link displayed to run the container. Note that the URL will be different for every new instance:
    Figure 6.34: Invoking the hello-world app
  7. Next, we are going to deploy a new revision of the application by updating the TARGET environment variable. Navigate back to the GCP console and click on the DEPLOY NEW REVISION button.
  8. From the Deploy revision to hello-world (us-central1) form, click on the SHOW OPTIONAL REVISION SETTINGS link, which will point us to the additional setting section:
    Figure 6.35: Optional revision settings
  9. Under the environment variables section, create a new environment variable named TARGET with the value Cloud Run Deployment:
    Figure 6.36: Setting the TARGET environment variable
  10. Click on the DEPLOY button.
  11. Now we can see the new revision of the hello-world application called hello-world-00002 with 100% of traffic being routed to the latest revision:
    Figure 6.37: The hello-world app's new revision
  12. Click on the URL again to run the updated revision:
Figure 6.38: Invoking the hello-world app

We have successfully deployed a pre-built Docker image on the Google Cloud Run platform.

Introduction to Virtual Kubelet

Virtual Kubelet is an open source implementation of Kubernetes' kubelet that acts as a kubelet. This is a sandbox project from the Cloud Native Computing Foundation (CNCF), and the first major version (v 1.0) of Virtual Kubelet was released on July 8, 2019.

Before diving further into Virtual Kubelet, let's recap what a kubelet is in the Kubernetes architecture. A kubelet is an agent that runs on each node in a Kubernetes cluster and is responsible for managing pods within the nodes. A kubelet takes instructions from the Kubernetes API to identify the pods to be scheduled on the node and interacts with the underlying container runtime (for example, Docker) of the nodes to ensure that the desired number of pods are running and that they are healthy.

In addition to managing pods, the kubelet performs several other tasks:

  • Updating the Kubernetes API with the current status of the pods
  • Monitoring and reporting node health metrics such as CPU, memory, and disk utilization to the Kubernetes master
  • Pulling Docker images from the Docker registry for the assigned pods
  • Creating and mounting volumes for pods
  • Providing an interface for the API server to execute commands such as kubectl logs, kubectl exec, and kubectl attach for the pods

The following figure displays a Kubernetes cluster with standard and virtual kubelets:

Figure 6.39: Kubernetes cluster with standard kubelets and Virtual Kubelets

Virtual Kubelet will appear as a traditional kubelet from the viewpoint of the Kubernetes API. This will run in the existing Kubernetes cluster and register itself as a node within the Kubernetes API. Virtual Kubelet will run and manage the pods in the same way a kubelet does. But in contrast to the kubelet, which runs pods within the nodes, Virtual Kubelet will utilize external services to run the pods. This connects the Kubernetes cluster to other services such as serverless container platforms. Virtual Kubelet supports a growing number of providers, including the following:

  • Alibaba Cloud Elastic Container Instance (ECI)
  • AWS Fargate
  • Azure Batch
  • Azure Container Instances (ACI)
  • Kubernetes Container Runtime Interface (CRI)
  • Huawei Cloud Container Instance (CCI)
  • HashiCorp Nomad
  • OpenStack Zun

Running pods on these platforms come with the benefits of the serverless world. We do not have to worry about the infrastructure as it is managed by the cloud provider. Pods will scale up and down automatically based on the number of requests received. Also, we have to pay only for the utilized resources.

Exercise 20: Deploying Virtual Kubelet on AKS

In this exercise, we are going to configure Virtual Kubelet on Azure Kubernetes Service (AKS) with the ACI provider. For this exercise, we will be using the following services available in Azure.

  • AKS: AKS is a managed Kubernetes service on Azure.
  • ACI: ACI provides a managed service for running containers on Azure.
  • Azure Cloud Shell: An interactive, browser-based shell that supports both Bash and PowerShell.

You need to have the following prerequisites for this exercise:

  • A Microsoft Azure account
  • The Azure CLI
  • The kubectl CLI
  • Helm

We will be using Azure Cloud Shell, which has all the previously mentioned CLIs pre-installed:

  1. Navigate to https://shell.azure.com/ to open Cloud Shell in a browser window. Select Bash from the Welcome to Azure Cloud Shell window:
    Figure 6.40: The Welcome to Azure Cloud Shell window
  2. Click on the Create storage button to create a storage account for Cloud Shell. Note that this is a one-time task purely for when we are using Cloud Shell for the first time:
    Figure 6.41: Mounting storage for Cloud Shell

    The Cloud Shell window will look as follows:

    Figure 6.42: Cloud Shell window
  3. Once Cloud Shell is ready, we can start creating the AKS cluster.

    First, we need to create an Azure resource group that allows us to group related Azure resources logically. Execute the following command to create a resource group named serverless-kubernetes-group in the West US (westus) region:

    $ az group create --name serverless-kubernetes-group --location westus

    The output should be as follows:

    Figure 6.43: Creating an Azure resource group
  4. Register your subscription to use the Microsoft.Network namespace:

    $ az provider register --namespace Microsoft.Networks

    The output should be as follows:

    Figure 6.44: Registering the subscription
  5. Next, we will create an Azure Kubernetes cluster. The following command will create an AKS cluster named virtual-kubelet-cluster with one node. This command will take a few minutes to execute:

    $ az aks create --resource-group serverless-kubernetes-group --name virtual-kubelet-cluster --node-count 1 --node-vm-size Standard_D2 --network-plugin azure --generate-ssh-keys

    Once AKS cluster creation is successful, the preceding command will return some JSON output with the details of the cluster:

    Figure 6.45: Creating the AKS cluster
  6. Next, we need to configure the kubectl CLI to communicate with the newly created AKS cluster. Execute the az aks get-credentials command to download the credentials and configure the kubectl CLI to work with the virtual-kubelet-cluster cluster with the following command:

    Note

    We are not required to install the kubectl CLI because Cloud Shell comes with kubectl pre-installed.

    $ az aks get-credentials --resource-group serverless-kubernetes-group --name virtual-kubelet-cluster

    The output should be as follows:

    Figure 6.46: Configuring kubectl
  7. Now we can verify the connection to the cluster from Cloud Shell by executing the kubectl get nodes command, which will list the nodes available in the AKS cluster:

    $ kubectl get nodes

    The output should be as follows:

    Figure 6.47: Listing Kubernetes nodes
  8. If this is the first time you are using the ACI service, you need to register the Microsoft.ContainerInstance provider with your subscription. We can check the registration state of the Microsoft.ContainerInstance provider with the following command:

    $ az provider list --query "[?contains(namespace,'Microsoft.ContainerInstance')]" -o table

    The output should be as follows:

    Figure 6.48: Checking the registration status of the Microsoft.ContainerInstace provider
  9. If the RegistrationStatus column contains a value of NotRegistered, execute the az provider register command to register the Microsoft.ContainerInstance provider. If the RegistrationStatus column contains a value of Registered, you can continue to the next step:

    $ az provider register --namespace Microsoft.ContainerInstance

    The output should be as follows:

    Figure 6.49: Registering for Microsoft.ContainerInstance provider
  10. The next step is to create the necessary ServiceAccount and ServiceAccount objects for the tiller. Create a file named tiller-rbac.yaml with the following code:

    apiVersion: v1

    kind: ServiceAccount

    metadata:

      name: tiller

      namespace: kube-system

    ---

    apiVersion: rbac.authorization.k8s.io/v1

    kind: ClusterRoleBinding

    metadata:

      name: tiller

    roleRef:

      apiGroup: rbac.authorization.k8s.io

      kind: ClusterRole

      name: cluster-admin

    subjects:

      - kind: ServiceAccount

        name: tiller

        namespace: kube-system

  11. Then execute the kubectl apply command to create the necessary ServiceAccount and ClusterRoleBinding objects:

    $ kubectl apply -f tiller-rbac.yaml

    The output should be as follows:

    Figure 6.50: Creating the ServiceAccount and ClusterRoleBinding objects
  12. Now we can configure Helm to use the tiller service account that we created in the previous step:

    $ helm init --service-account tiller

    The output should be as follows:

    Figure 6.51: Configuring tiller
  13. Once all configurations are done, we can install Virtual Kubelet using the az aks install-connector command. We will be deploying both Linux and Windows connectors with the following command:

    $ az aks install-connector \

        --resource-group serverless-kubernetes-group \

        --name virtual-kubelet-cluster \

        --connector-name virtual-kubelet \

        --os-type Both

    The output should be as follows:

    Figure 6.52: Installing Virtual Kubelet
  14. Once the installation is complete, we can verify it by listing the Kubernetes nodes. There will be two new nodes, one for Windows and one for Linux:

    $ kubectl get nodes

    The output should be as follows:

    Figure 6.53: Listing Kubernetes nodes
  15. Now we have Virtual Kubelet installed in the AKS cluster. We can deploy an application to a new node introduced by Virtual Kubelet. We will be creating a Kubernetes Deployment named hello-world with the microsoft/aci-helloworld Docker image.

    We need to add a nodeSelector to assign this pod specifically to the Virtual Kubelet node. Note that Virtual Kubelet nodes are tainted by default to prevent unexpected pods from being run on them. We need to add tolerations to the pods to allow them to be scheduled for these nodes.

    Let's create a file named hello-world.yaml with the following content:

         apiVersion: apps/v1

    kind: Deployment

    metadata:

      name: hello-world

    spec:

      replicas: 1

      selector:

        matchLabels:

          app: hello-world

      template:

        metadata:

          labels:

            app: hello-world

        spec:

          containers:

          - name: hello-world

            image: microsoft/aci-helloworld

            ports:

            - containerPort: 80

          nodeSelector:

            kubernetes.io/role: agent

            type: virtual-kubelet

            beta.kubernetes.io/os: linux

          tolerations:

          - key: virtual-kubelet.io/provider

            operator: Equal

            value: azure

            effect: NoSchedule

  16. Deploy the hello-world application with the kubectl apply command:

    $ kubectl apply -f hello-world.yaml

    The output should be as follows:

    Figure 6.54: Creating the hello-world deployment
  17. Execute the kubectl get pods command with the -o wide flag to output a list of pods and their respective nodes. Note that the hello-world-57f597bc59-q9w9k pod has been scheduled on the virtual-kubelet-virtual-kubelet-linux-westus node:

    $ kubectl get pods -o wide

    The output should be as follows:

Figure 6.55: Listing all pods with the -o wide flag

Thus, we have successfully configured Virtual Kubelet on AKS with ACI and have deployed a pod in the Virtual Kubelet node.

Let's now complete an activity where we will be deploying a containerized application in a serverless environment.

Activity 6: Deploy a Containerized Application in a Serverless Environment

Imagine that you are working for a start-up company and your manager wants you to create an application that can return the current date and time for a given timezone. This application is expected to receive only a few requests during the initial phase but will receive millions of requests in the long run. The application should be able to scale automatically based on the number of requests received without any modifications. Also, your manager does not want to have the burden of managing the infrastructure and expects this application to run with the lowest possible cost.

Execute the following steps to complete this activity:

  1. Create an application (in any language you want) that can provide the current date and time based on the given timezone value.

    The following is some sample application code written in PHP:

         <?php

    if ( !isset ( $_GET['timezone'] ) ) {

        // Returns error if the timezone parameter is not provided

        $output_message = "Error: Timezone not provided";

    } else if ( empty ( $_GET['timezone'] ) ) {

        // Returns error if the timezone parameter value is empty

        $output_message = "Error: Timezone cannot be empty";

    } else {

        // Save the timezone parameter value to a variable

        $timezone = $_GET['timezone'];

        

        try {

            // Generates the current time for the provided timezone

            $date = new DateTime("now", new DateTimeZone($timezone) );

            $formatted_date_time = $date->format('Y-m-d H:i:s');

            $output_message = "Current date and time for $timezone is $formatted_date_time";

        } catch(Exception $e) {

            // Returns error if the timezone is invalid

            $output_message = "Error: Invalid timezone value";

        }

    }

    // Return the output message

    echo $output_message;

  2. Containerize the application according to the guidelines provided by Google Cloud Run.

    The following is the content of a sample Dockerfile:

    # Use official PHP 7.3 image as base image

    FROM php:7.3-apache

    # Copy index.php file to the docker image

    COPY index.php /var/www/html/

    # Replace port 80 with the value from PORT environment variable in apache2 configuration files

    RUN sed -i 's/80/${PORT}/g' /etc/apache2/sites-available/000-default.conf /etc/apache2/ports.conf

    # Use the default production configuration file

    RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"

  3. Push the Docker image to a Docker registry.
  4. Run the application with Cloud Run.

    The output should be as follows:

Figure 6.56: Deployment of the application in a serverless environment

Note

The solution to the activity can be found on page 417.

Summary

In this chapter, we discussed the advantages of using serverless on Kubernetes. We discussed three technologies that offer the benefits of serverless on top of a Kubernetes cluster. These are Knative, Google Cloud Run, and Virtual Kubelet.

First, we created a GKE cluster with Istio and deployed Knative on top of it. Then we learned how to deploy an application on Knative. Next, we discussed the serving component of Knative and how to perform a canary deployment with configuration and route objects. Then we discussed monitoring on Knative and observed how Knative autoscaling works based on the number of requests received.

We also discussed Google Cloud Run, which is a fully managed platform, built on the Knative project, to run stateless HTTP-driven containers. Then we learned how to deploy an application with the Cloud Run service.

In the final section, we studied Virtual Kubelet, which is an open source implementation of Kubernetes' kubelet. We learned the differences between normal kubelets and Virtual Kubelet. Finally, we deployed Virtual Kubelet on an AKS cluster and deployed an application to a Virtual Kubelet node.

In the next three chapters, we will be focusing on three different Kubernetes serverless frameworks, namely Kubeless, OpenWhisk, and OpenFaaS.