Planning for IBM Spectrum Scale CSI driver deployment
There are several components that must be installed and setup before deploying and using IBM Spectrum Scale CSI driver. An understanding of various deployment mechanisms and topologies of these components will help meet the pre-requisites for IBM Spectrum Scale CSI driver.
In this chapter we cover the following topics:
This chapter is recommended for all administrators.
4.1 Deployment on Kubernetes
This section describes the IBM Spectrum Scale components that need to be installed on each of the Kubernetes cluster nodes:
•Worker nodes: You need to install IBM Spectrum Scale client and IBM Spectrum Scale CSI driver on the Worker nodes on which you need access to persistent volumes.
•Infrastructure nodes: You need to install IBM Spectrum Scale client and IBM Spectrum Scale CSI driver on infrastructure nodes, such as Attacher node or Provisioner node. The infrastructure node must be able to communicate with IBM Spectrum Scale GUI server in the IBM Spectrum Scale cluster.
•Master node: Do not include this node as part of IBM Spectrum Scale cluster.
See Figure 4-1 for typical deployment on a Kubernetes cluster. the following are recommendations for IBM Spectrum Scale cluster nodes:
•GUI nodes, protocol nodes, and AFM nodes should be configured outside of the Kubernetes cluster.
•NSD nodes are to be outside of Kubernetes cluster but can be included in Kubernetes cluster for Shared Nothing Cluster (SNC) setup.
•IBM Spectrum Scale GUI node is a prerequisite for the IBM Spectrum Scale CSI driver. The IBM Spectrum CSI Driver uses the GUI REST API feature to perform all of its operations. Currently, CSI Driver can be configured to use only a single GUI node in IBM Spectrum Scale Cluster.
Figure 4-1 Typical deployment on a Kubernetes cluster
See IBM Knowledge Center section IBM Spectrum Scale Container Storage Interface Driver for additional deployment considerations.
4.1.1 Kubernetes deployment mechanism
Kubernetes can be deployed with various tools described in this section and the majority used for Kubernetes deployment methods along with components such as container runtime and network plug-ins.
IBM Spectrum Scale CSI driver is tested with Kubeadm, Kubespray, and Minikube. You can get more information about each of these methods using the following links:
4.1.2 Container runtime
A container runtime is software that executes containers and manages container images on a node. There are many container runtimes available. However, currently the most widely used container runtime is Docker. IBM Spectrum Scale CSI driver is tested with Docker and CRI-O. You can use the following links for more information on the container runtime:
4.1.3 Network plug-in
Container networking supports many network plug-ins currently available. Typically, Kubernetes use Calico and OpenShift uses OVS. The IBM Spectrum Scale CSI driver has been tested with Calico and OVS.
4.2 Deployment on OpenShift
OpenShift 4 introduces a new way of installing the platform that is automated, reliable, and repeatable. Based on the Kubernetes Cluster-API SIG, Red Hat has developed an OpenShift installer for full stack automated deployments. With this release, the installer not only installs OpenShift, but it installs (and manages) the entire infrastructure as well, from DNS all the way down the stack to the VM. This provides a fully integrated system that can resize automatically with the needs of your workload. Currently, full stack automated deployment is supported on AWS. Currently, guides for pre-existing infrastructure installs exist for AWS, VMWare vSphere, and bare metal.
IBM Spectrum Scale CSI is tested with Kubernetes Standard Deployment, OpenShift with Bare Metal, VMware vSphere, and AWS.
See Figure 4-2 for the IBM Spectrum Scale CSI driver deployment architecture on OpenShift.
Figure 4-2 IBM Spectrum Scale CSI driver deployment on OpenShift
OpenShift on Bare Metal
OpenShift Bare Metal requires a minimum of one bootstrap machine; three control planes, or master, machines; and at least two computes, or worker, machines. IBM Spectrum Scale is deployed on worker nodes.
For more detail about an OpenShift Bare Metal install, see the following link:
The OpenShift deployment master nodes are on CoreOS and worker nodes are on RHEL. IBM Spectrum Scale and CSI are deployed on a worker node.
OpenShift on vSphere
Red Hat and VMware are collaborated to integrate OpenShift Container Platform and Funware’s software-defined data center (SDDC) infrastructure stack. VMware and Red Hat both embrace Kubernetes as the core platform supporting their modern applications. IBM Spectrum Scales deployed on worker nodes.
For more Information about installing OpenShift on vSphere, see the following link:
4.3 IBM Spectrum Scale deployment
In order for the CSI driver to function optimally, it is important to pay attention to some of the deployment aspects of IBM Spectrum Scale.
4.3.1 IBM Spectrum Scale deployment models
The IBM Spectrum Scale CSI driver can be used with IBM Spectrum Scale cluster that’s configured to use Shared storage or NSD Server model. The IBM Spectrum Scale CSI driver can also be used in IBM Spectrum Scale configurations that use ESS storage; but Kubernetes, OpenShift, and CSI components cannot be installed on ESS I/O nodes.
4.3.2 Network considerations
The following network guidelines need to be considered during IBM Spectrum Scale CSI driver planning and implementation:
•The infrastructure node must be able to communicate with IBM Spectrum Scale GUI server in the Scale cluster
•If you are using storage from remote IBM Spectrum Scale cluster, the infrastructure nodes must be able to communicate with GUI servers in local IBM Spectrum Scale cluster and remote IBM Spectrum Scale cluster
•In case of higher performance requirement, you can configure the Kubernetes worker nodes and NSD server nodes in the Storage cluster on a high speed network.
4.3.3 Primary File System
During IBM Spectrum Scale CSI driver initialization one cluster must be designated as the primary cluster, and one file system within that cluster must be designated as the primary file system in the CSI driver configuration.
The CSI driver can be configured to use multiple IBM Spectrum Scale clusters in a remotely mounted cluster relationship but the primary cluster is the one whose nodes are part of the Kubernetes cluster. There can only be one cluster that can be used as a primary cluster.
The primary file system is a file system within the primary cluster that is used for storing links to all PVCs. This file system must be created by the IBM Spectrum Scale administrator before initializing the CSI driver. It must be mounted on all required worker nodes and the GUI node. A primary file set is created by the CSI driver within this file system if it doesn’t exist. When PVCs are provisioned, softlinks to the corresponding directories/file sets are created inside this file set.
Other file systems can be created in IBM Spectrum Scale cluster anytime before or after CSI driver initialization, and can be used in the storageclass parameter for PVC provisioning. See Chapter 5, “Deployment and Administration” on page 29 for further details on configuration.
4.4 What type of volumes do I need
IBM Spectrum Scale CSI driver supports following three types of volumes
– A new directory is created for every new volume provisioned dynamically using lightweight volume storageclass
– Suitable for creating a large number of volumes (>10000)
– PVC size cannot be imposed on storage due to lack of directory level quota support
•Independent file set based volumes
– A new independent file set is created for every new volume provisioned dynamically using independent file set storageclass
– Suitable when the number of volumes required is less than 1000
– PVC size can be honored on storage by means of file set quota
– IBM Spectrum Scale administrator can leverage management benefits of file sets on these volumes, such as snapshots and backup
•Dependent file set based volumes
– A new dependent file set is created for every new volume provisioned dynamically using dependent file set storageclass
– Suitable when number of volumes required is less than 10000
– PVC size can be honored on storage by means of file set quota
There are two ways in which volumes can be provisioned
•Dynamic provisioning: The resource is created on IBM Spectrum Scale file system dynamically during PVC creation.
•Static provisioning: The resource is pre-existing on IBM Spectrum Scale file system and can be used to define a PV.
4.5 Limitations of IBM Spectrum Scale CSI driver
For limitations of IBM Spectrum Scale CSI driver see the following IBM Knowledge Center document: