Chapter 5: Assessing Security in the Cloud – Cloud Computing: Assessing the Risks


In Chapter 4, we discussed why Cloud Computing security is different from traditional approaches to computing security. Among the reasons cited were:

  • Virtualisation  Many traditional security solutions rely on examining network traffic. In virtualised environments, network traffic often goes from one virtual machine to another without leaving the physical server, rendering network-attached security devices ineffective.
  • Dynamic environments  Virtualisation environments support dynamic placement and relocation of virtual machines to enable hardware failure resiliency and better application performance. The side effect of this is that security practices that assume a static environment are challenged to operate effectively in a dynamic infrastructure.
  • Multi-tenant environments  Many security products and practices were designed for environments that are controlled by a single entity. In such environments, examining all network packets or performing port scanning is perfectly acceptable. In a multi-tenant Cloud environment, such approaches are often a violation of the provider’s terms of service.

In Cloud Computing environments, the reality is that security is no longer the duty of a single entity. Instead, security is a shared responsibility: part of the responsibility lies with the Cloud provider, and part lies with the Cloud user. The question is, how does one assess the security of those portions of the Cloud Computing environment that lie within the responsibility area of the provider?

Assessing Cloud security

One approach, of course, would be to rely on the word of the provider. The provider would assert that everything in the environment is fine, security-wise, and the user would accept the assertion as definitive.

The problem with that approach is that, while the user would rely on the provider to implement security correctly, nearly all of the risk for any security issue falls upon the user of the service. If you look at 7Figure 1, first introduced in Chapter 2, you’ll note that its implication is quite clear: while providers and users share responsibility for security and compliance, risk associated with failure lies primarily with the user.

In some sense, it may seem unfair that Cloud users bear risk responsibility for infrastructure elements out of their control, but that is the reality of how most Cloud service contracts are drawn up. However, just to reinforce a point made earlier, this asymmetric risk responsibility is not unique to Cloud Computing. Nearly every type of technology outsourcing contract is carefully drawn up by the provider to limit its risk exposure. In fact, one might infer that these contracts are deliberately drawn up to minimise the risk providers face and to shield themselves from any consequences that might arise from their actions.

Given this state of affairs, fair or not, users must recognise that the quality of the provider’s security is important to them, and measures beyond blind faith are required. Failing to implement measures designed to minimise security, or compliance issues located within the provider’s area of responsibility, raises the likelihood that the user will increase its overall risk.

Peeking below the trust boundary

Chapter 4 also further discussed the concept of a ‘trust boundary’ (see Figure 6). The trust boundary represents the interface that identifies where a user’s responsibility for direct security implementation ends and where the provider’s begins. If you look at the figure, you’ll note that the actions identified below the trust boundary are ‘evaluation and certification’.

What this means is that, in the absence of the ability to perform direct security actions (e.g. place a traffic sniffer on the internal data centre network), one must evaluate the measures the provider takes to secure the areas of its responsibility.

On its face, evaluation is a very clear concept: one observes the actions taken by another party and considers whether they are sufficient to implement the objectives of the observer. This practice is present in every situation in which one relies on another party to perform activities. For example, when one calls a plumber to come and fix a leaking pipe, before paying them, one evaluates whether the plumber has implemented fixing the pipe correctly.

The challenge of evaluation

Evaluation, however, can be a complex task. Even for something that, at first sight, seems like a straightforward situation to assess, evaluation can be very difficult. Take as an example, the plumbing evaluation just mentioned. While it might seem trivial to assess whether a plumber fits a pipe properly, even that task carries with it complexities like:

  • Was the task performed correctly? It might not leak now, but did the plumber use techniques that will prevent leaks in the future? Did they use sealant that will stand up over time, or did they use cheap sealant that will break down quickly?
  • Did they follow all applicable rules and regulations? Even for something as simple as a pipe (basically, a tube that allows a fluid to be sent from one place to another – what could be simpler?), there can be a number of regulations that are applicable – size, pipe composition, bracing (if necessary) and so on.
  • Do they have all necessary licences and certifications? While many deride the over-regulation that occurs in many domains, licences and certifications are a fact of life. And the lack of a plumber’s necessary licensing might become important in the event of an insurance claim or lawsuit.

Consequently, evaluation can be a very important and difficult proposition. And many organisations (and homeowners, in the case of plumbing!) find it difficult to conduct an appropriate evaluation. Some of the reasons are:

  • Lack of domain expertise  While it’s easy to discern whether a pipe is leaking after the plumber concludes their work, a homeowner may not know all the applicable laws and regulations. For more complex matters like Cloud Computing, organisations may not have the knowledge base to discern whether a Cloud provider is doing a good job or not.
  • Applying expertise to a new domain  Even detailed knowledge about something fairly similar to a given domain may be insufficient for a new domain. Knowing a lot about water-oriented plumbing may not help in knowing enough about natural gas plumbing. In the case of Cloud Computing, knowing a lot about running a data centre may not be sufficient to understand compliance needs and best practices for operating a multi-tenant data centre environment.
  • Creating time to conduct a thorough evaluation  Everyone is busy, and IT organisations have been systematically squeezed of headcount. Most would find it difficult to divert staff to conduct a thorough Cloud security evaluation, just as most of us would find it difficult to find enough time, given our hectic lives, to fully assess the quality of a plumber’s work.
  • Getting attention from the vendor  How can you get a rich enough interaction to ensure your evaluation obtains sufficient information to make the necessary assessment? It’s an unfortunate fact of life that Cloud providers pay attention to their largest customers and minimise interaction with customers who represent small revenue opportunities – even if a thorough assessment is critical for that small opportunity customer. To be fair to Cloud providers, it’s challenging for them to deal with an environment in which every customer or potential customer wants to conduct an evaluation. Every evaluation requires time and attention from provider personnel, and most of the evaluations will, ultimately, be nearly identical in 90% of the aspects they assess. Clearly, this repetitive evaluation process is inefficient. There should be a better way. Fortunately, there is.

Role of certification

Many industries confront the evaluation problems outlined above. How can buyers assess the quality of industry vendors, given the complexities and time constraints present in all businesses? How can vendors reduce the time devoted to evaluation in an environment in which many buyers want to perform similar evaluation processes?

The solution for most industries is to develop a set of recommended best practices promulgated by an impartial trade association, usually comprised of vendor and user representatives. The recommended best practices are typically characterised as a standard, indicating that all vendors that wish to be recognised as high quality must meet the requirements laid out in the standard.

Because IT systems often have financial implications due to their use in company operations, or as the system of record for financial transactions, the accounting industry often participates in these association efforts.

Alternatively, the accounting industry may itself develop and promulgate the recommended practices. In doing so, it would ordinarily create a body tasked with developing the standard; this body would include representatives from all interested parties, including the accounting industry, vendors, users and perhaps government agencies and regulatory bodies.

Once a standard is in place, it simplifies the evaluation process. Instead of each participant in an industry developing its own criteria, everyone can use the standard as the basis for evaluation.

Of course, having a common set of criteria doesn’t solve the problem of each user conducting its own evaluation. If each potential customer seeks to conduct its own evaluation against the criteria, there is clearly an issue with redundancy of effort, which is inefficient and expensive. Moreover, the existence of a standard doesn’t necessarily solve the problem of lack of user expertise – just because criteria have been established, doesn’t mean that users have the knowledge base to understand the criteria or the judgement to assess a vendor’s compliance with the standard.

Certifications and audits

Consequently, the standards process has taken the further step of establishing an audit process, designed to standardise the evaluation process.

A vendor can undergo an audit, whereby an external party will assess the vendor’s compliance with the relevant standard, and pass judgement as to the level of compliance present in the vendor’s environment.

The audit process simplifies things enormously. Instead of repetitive evaluations conducted by multiple customers, the vendor can undergo one evaluation process, at the end of which it receives a certificate of compliance with the standard. Any subsequent customer who wishes to establish the vendor’s compliance with the standard can accept the certification as proof that the vendor meets the standard’s requirements.

Of course, this raises the question: who is able to conduct such an audit and how can that organisation be trusted to do a good job?

The answer, naturally enough, is that the body that promulgated the standard certifies the auditing organisation. The auditor goes through a process designed to ensure that it has the expertise and thoroughness to fully evaluate a vendor’s compliance with the standard. As the end of the process, the auditor is certified as being capable of performing audits against the standard and, crucially, able to provide the vendor being evaluated with a certificate of standard compliance.

How certifications work

Certification refers to a process whereby a provider has an external auditor examine a given domain and evaluate it against a formal set of criteria. These criteria are typically created by an independent industry body, thereby assuring objectivity and impartiality in the criteria definition.

The external auditor gains access to the provider’s infrastructure and assesses its practices according to the criteria defined in the particular certification domain. After the audit is complete, if the provider’s practices meet the requirements of the audit, the auditor issues a report detailing the provider’s compliance with the certification criteria. The shorthand phrase for this process is that the provider is certified against a particular audit process.

Once a provider has undergone the audit process and received a certification, it can present itself as certified for a particular domain. In turn, customers or potential customers of that provider can accept the certification in place of performing their own audit.

As can be seen, the certification process neatly addresses the issues regarding user evaluation identified earlier in this chapter. Rather than attempting to create and perform its own audit, a customer or potential customer can identify the key audit/certification domains relevant to its needs and request those from service providers.

Many companies considering using an external provider go even further: they insist that providers, in order to be considered as a potential choice, must have undergone specific audit processes and offer proof of successfully achieving certification.

As one can easily imagine, the ability to rely on impartial audit requirements and associated certificates can simplify the evaluation process enormously.

Common certification processes germane to Cloud Computing include:

  • COBIT (IT Governance and Control)  This set of requirements relates to how IT organisations are managed, including how internal processes are defined and enforced.
  • HIPAA (Health Insurance Portability and Accountability Act)  This Act sets out a number of requirements relating to information privacy. As suggested by its name, HIPAA is focused specifically on health care and affects organisations of many different types involved in the medical field.
  • SP 800-53  This NIST (National Institute of Standards and Testing) Special Publication focuses on security controls for IT environments. NIST published this as a guide to help government organisations prepare to undergo audits performed under the Federal Information Security Management Act (FISMA). While the titles of the organisation and the Act suggest this affects only government bodies, service providers that seek to host federal government applications need to meet FISMA requirements, so this standard is germane to CSPs, as well as government agencies.
  • FedRAMP  The Federal Risk and Authorization Management Program is an initiative sponsored by the federal government to make the audit process easier for government agencies and CSPs. The FISMA standard outlined above requires every application to undergo a security audit, part of which assesses the infrastructure upon which the application resides. Obviously, if a large number of applications need to be audited in a process that is largely repetitive with respect to the infrastructure, this is inefficient and costly. FedRAMP is an effort to streamline this process and enable a single audit process to be used by multiple FISMA efforts.
  • PCI-DSS  The Payment Card Industry Data Security Standard is a standard associated with taking electronic payments. Obviously, there are important requirements in the area of security and privacy with respect to financial transactions, and PCI sets out the requirements in this domain.
  • ISO27001 (full title: ISO/IEC 27001:2005 Information technology – Security techniques – Information security management systems – Requirements)  This standard, published by the International Standards Organisation, sets out security controls designed to ensure that IT organisations have a defined and consistent approach to information security.
  • BITS  BITS is part of the Financial Institution Shared Assessments Program and is designed to help assess the security controls of IT service providers. From the name of the sponsoring organisation, it is obvious that this is oriented toward the financial services industry. This initiative is analogous to the FISMA and FedRAMP audit standards described above.
  • SAS 70 (full title: Statement on Auditing Standards (SAS) No. 70, Service Organisations)  Note that SAS 70 was superseded by SSAE 16 (full title: Reporting on Controls at a Service Organisation) in mid-2011 and will be the appropriate certification going forward. SAS 70 and its successor provide an assessment of the processes a provider follows in delivering a service.. This assessment ensures that processes are defined explicitly and that the provider has measures in place to ensure that they are consistently executed.
  • GAPP  The Generally Accepted Privacy Principles are a set of measures formulated by the American Institute of CPAs (AICPA) directed toward IT privacy practices and policies. As can be seen from its title, GAPP is directed toward some of the same objectives as HIPAA.

Dealing with multiple compliance standards

There is an old joke in the IT industry that standards are great and that’s why there’s so many of them. The joke illustrates what must be obvious from the list in the last section: there are lots of compliance requirements, some generally applicable and others focused on specific industries. Intuitively, there must be significant overlap among the different compliance regimes.

Consequently, while in the abstract the existence of audit requirements and certifications seems that it would make life simple for IT users, it would be more appropriate to say that it makes life simpler.

While audits and certifications may address the detailed specifics of security and privacy, which is certainly a huge benefit, they still present an IT organisation with the need to assess which compliance standards are appropriate for its specific environment.

Moreover, if multiple compliance requirements are germane for the IT organisation, the obvious question is how to ensure that every element of each requirement is addressed without having to repetitively run through each audit standard.

Finally, even if an IT organisation is able to understand how to manage each of the audit standards appropriate to its specific situation, it is still faced with the need to understand how to apply them to a CSP’s Cloud offering.

Cloud Security Alliance

Fortunately, companies seeking to sort out Cloud Computing security and compliance don’t have to fight the battle alone. The Cloud Security Alliance (typically referred to as CSA) is an organisation that provides a locus of research, expertise and recommendations in the area of Cloud security. The CSA is located at

Founded in 2008, the CSA is a not-for-profit organisation with a mission ‘to promote the use of best practices for providing security assurance within Cloud Computing, and provide education on the uses of Cloud Computing to help secure all other forms of computing’.

The CSA is the centre of gravity with respect to the topic of Cloud Computing security. It is comprised of end-users, vendors and service providers, all of whom interact in the interest of defining what Cloud Computing security requires and how to achieve it.

CSA has an international scope, and has both national and local chapters. Anyone seeking to understand the nuances of Cloud security and to learn current best practices should consider getting involved in the CSA. It sponsors a yearly Cloud security conference in November; if you are someone seeking deeper involvement in Cloud Computing security, this conference is a must-attend event.

Leveraging the CSA

In addition to its overall security focus, the CSA has published research documents focused on easing the audit and compliance complexity faced by Cloud users.

As a general overview of how security and compliance relate to Cloud Computing, the CSA published the ‘Guidance for Critical Areas of Focus in Cloud Computing’. This document discusses how Cloud Computing relates to 13 different security and compliance areas, including governance and risk management, interoperability and portability, and data centre operations, to name but three.

The ‘Guidance’ document provides an excellent overview and jumping-off point to better understand the landscape of Cloud Computing security.

With respect to specific aspects of assessing the security of a CSP, two further CSA documents are highly valuable for IT organisations. The documents also provide guidance on how to map the most relevant audit standards to one another.

The first document is the Consensus Assessments Initiative Questionnaire (CAI), published in late 2010. This questionnaire ‘provides a set of questions a Cloud consumer and Cloud auditor may wish to ask of a Cloud provider. It provides a series of yes or no control assertion questions which can then be tailored to suit each unique Cloud customer’s evidentiary requirements.’

The second document is The CSA Cloud Controls Matrix (CCM), which provides a ‘controls framework that gives detailed understanding of security concepts and principles that are aligned to the Cloud Security Alliance guidance in 13 domains’. The CCM was first published in April 2010, and has subsequently been updated, with the latest version released in August 2011.

All three of these documents may be found at

Overview of the CAI and CCM

The purpose of both the CAI and CCM is to provide guidance and recommendations in a number of specific areas for IT organisations wishing to audit CSPs and assess their security practices.

The two documents share a common taxonomy of security aspects, which makes it easy to cross-correlate the security requirements (the controls) to assess how well the CSP implements them (the questionnaire).

The security aspects of the two documents are as follows:

  • compliance
  • data governance
  • facility security
  • human resources
  • information security
  • legal
  • operations management
  • risk management
  • release management
  • resiliency
  • security architecture.

Each of these areas has one or more individual items associated with it. For example, release management has five separate items:

  • new development/acquisition
  • production changes
  • quality testing
  • outsourced development
  • unauthorised software installations.

The CCM outlines the control requirements for each item in a matrix. These requirements are assertions about how a CSP should comply with the individual item.

The CAI contains the same areas and items, but lists a set of questions to be posed to a CSP for each item to assist in determining whether the CSP meets the requirements outlined in the CCM. This list of questions helps IT organisations formulate an organisation-specific audit – they provide the foundation for a complete set of questions that will reflect the organisation’s audit and compliance requirements.

A number of compliance standards relevant to Cloud Computing were outlined earlier in this chapter. As noted, some of them overlap. Furthermore, for an individual IT organisation, figuring out which parts of each standard are relevant and how they relate to the CSA’s security areas and items would seem challenging, to say the least.

Fortunately, the CSA has made this challenge much simpler. Both the CAI and CCM contain columns for each security standard. For every security item outlined in the documents, the relevant portions of each standard are called out. For example, for the item ownership/stewardship within the data governance area, the COBIT column notes as relevant areas COBIT 4.1 DS5.1 and PO 2.3.

The benefit this set of columns provides for IT organisations is immense. Rather than having to work through each relevant compliance standard and assess which part of each standard is appropriate to different audit areas, the organisation can rely on the list the CSA has put together. This simplifies the audit task enormously. Of course, it is still necessary to interact with individual CSPs and work through the audit process, but an evaluation framework makes the audit process easier.

The CSA and its CAI and CCM provide an excellent set of resources for the very important security issues associated with using a CSP. The CAI and CCM are, as just noted, a foundation for the audit process itself. Naturally, individual organisations may find it necessary to identify additional audit items or may be subject to additional compliance standard requirements, but the two documents offer an excellent starting point.

Mapping the CAI and CCM to the security stack

For many people, particularly those for whom security is not a primary focus or strength, the organisation of the CAI and CCM may be problematic. While the taxonomy areas of the documents may make sense to security professionals, IT personnel in the applications and infrastructure/operations areas may be more comfortable evaluating security in terms of a security stack, much like the one presented in Figure 6 of Chapter 4.

This poses a problem since the items within the CAI and CCM are organised topically (i.e. information governance) and not according to where each item falls within the architecture stack. For IT organisations that assign security responsibility according to which part of the stack the security item resides, a mapping of the CSA areas and items to the stack is useful.

Table 1 lists each part of the security stack and identifies which item(s) from the CAI and CCM are associated with it. This structure will aid IT organisations as they work through audit processes and assign individuals to participate in the process.

Table 1

Security layer

Consensus assessment
initiative section(s)
General security policyCompliance 01–08
 Legal 01–02
 Human resources 01–03
 Information security 01–16, 22–27, 32
 Operations management 01
 Release management 01
 Resiliency 01–02
 Risk management 01–05
Security 01
Facility and facility computing infrastructureFacility security 01–08
Resiliency 03–08
Facility Computing InfrastructureData governance 01–08
 Information security 04, 09–10, 18–19, 23, 28–32
 Operations management 02–03, 05
 Risk management 05
 Release management 01
Security 03, 05, 10–13,
Facility Software InfrastructureInformation security 04, 07–10, 17 20–21, 29, 32–34
 Release management 02–05
 Security 02, 04, 06–09, 14–15
 Operations management 04


This chapter has focused on evaluating CSP security. It began by identifying why virtualisation and Cloud Computing present challenges to established IT security practices.

As noted, virtualisation itself poses challenges to traditional security practices. Abstracting dependence upon physical devices offers significant operational and financial benefits, but can prevent common practices like packet inspection from operating properly.

The concept of the trust boundary was introduced, with the corresponding fact that using a CSP prevents direct application of many common security solutions. For example, lacking control of the computing environment, security groups cannot take advantage of solutions like placing intrusion detection and prevention appliances on the network.

The existence of the trust boundary means that IT organisations must rely on evaluation and certifications when it comes to assessing the security of a CSP. Understanding, not to say implementing, a CSP audit process is in itself a challenge, with problems posed by the need to develop Cloud-specific knowledge, as well as understanding and interpreting individual compliance standards.

Fortunately, organisations recognising the challenges caused by Cloud Computing security came together to form the Cloud Security Alliance. Beyond offering a gathering point for users, vendors and service providers to collaborate in this area, the CSA has published research documents offering guidance on evaluating CSP security.

Two relevant CSA documents were discussed: the Consensus Assessments Initiative and the Cloud Controls Matrix. Use of these documents can simplify and accelerate evaluation of a CSP’s security practices and also build confidence on the part of users that the CSP has implemented appropriate practices.

Finally, for those for whom a security evaluation organised according to a stack architecture is more comfortable, a mapping from the security stack first presented in Chapter 4 to the CSA evaluation documents was presented in table format.

Using these tools, one’s confidence in a CSP’s security practices can be increased. For many, this would seem to complete the topic of Cloud security – for them, all security issues rest with the provider and are the provider’s responsibility.

Of course, that is too simple and convenient a position. Most providers, while implementing (and demonstrating, via the evaluation process described in this chapter) appropriate security practices, unmistakeably reduce their risk exposure, leaving the majority of risk for any security issues with the user.

Beyond the obvious business motivation for this – it makes perfect sense to reduce one’s exposure to risk – there is another very good reason why CSPs insist on risk limitation. The reason is that the CSP’s Cloud environment does not exist in isolation – there is always an application that is part of the equation. And no CSP can (nor should it) take on responsibility for the security of a user’s application. No user wants to allow a third party (i.e. the CSP) to rifle through its code or data; consequently, the CSP quite rightly disavows any responsibility for the security of the application. Application security is explicitly left to the user, which causes its own set of problems.

Just as infrastructure security practices must change in a CSP setting, so too must application security practices. Many IT organisations fail to understand this, to their peril. Identifying and implementing appropriate application security practices is critical in a Cloud Computing environment, and it is to that subject we turn in the next chapter.