Chapter 8: Certifications Past and Future: A Future Model for Assigning Certifications that Incorporate Lessons Learned from Past Practices – Assured Cloud Computing

Certifications Past and Future: A Future Model for Assigning Certifications that Incorporate Lessons Learned from Past Practices

Masooda Bashir1, Carlo Di Giulio2,3, and Charles A. Kamhoua4

1School of Information Sciences, University of Illinois at Urbana-Champaign, Champaign, IL, USA

2Information Trust Institute, University of Illinois at Urbana-Champaign, Urbana, IL, USA

3European Union Center, University of Illinois at Urbana-Champaign, Champaign, IL, USA

4U.S. Army Research Laboratory, Network Sciences Division, Network Security Branch, Adelphi, MD, USA

Security certifications are widely used to demonstrate compliance with privacy and security principles, but over the last few years, new technologies and services – such as cloud computing applications – have brought new threats to the security of information, making existing standards weak or ineffective.

Three of the most highly regarded information technology security certifications used to assess cloud security are ISO/IEC 27001, SOC 2, and FedRAMP. ISO and SOC 2 have been used worldwide since 2005 and 2011, respectively, to build and maintain information security management systems or controls relevant to confidentiality, integrity, availability, security, and privacy within a service organization; FedRAMP was created in 2011 to meet the specific needs of the U.S. government in migrating its data on cloud environments.

This chapter describes the evolution of these three security standards and the improvements made to them over time to cope with new threats, and focuses on their adequacy and completeness by comparing them to each other. Understanding their evolution, resilience, and adequacy sheds light on their weaknesses and thus suggests improvements needed to keep pace with technological innovation.

8.1 Introduction

As cloud computing becomes more and more pervasive in our lives, and cloud services evolve to be easily available, flexible, and usable, their presence is accompanied by concerns for privacy and security. Cloud computing consists in “ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources” [1] built on one of three service models working on infrastructure maintained by a cloud service provider (CSP). The first model is called software as a service (SaaS) and allows access to predefined software applications managed by the CSP. The second is platform as a service (PaaS), which allows users to deploy supported software applications on the CSP's infrastructure. The third is infrastructure as a service (IaaS), which allows greater flexibility and deployment of arbitrary software [1].

The great success of cloud computing is based in its flexibility and the unprecedented possibilities offered by access to scalable, potentially unlimited storage and computing resources at a relatively low cost. The cloud represents a completely new approach to provisioning of and access to information technology resources. On the one hand, CSPs can leverage economies of scale, and the more their services are accessed and used and their capabilities expanded, the higher their performance or the lower their price for the same service. On the other hand, users can rely on commercial services for processing their business-related information, thus completely or partially removing their costs for maintenance and configuration of hardware and software; further, the cheaper the service, the more convenient it becomes to abandon the old “on-premises” model [2,3]. These characteristics were determinant for the U.S. federal government's adoption of a coordinated set of initiatives aimed to reallocate investment in information technology (IT)1 from the building and maintenance of on-premises infrastructure to the on-demand access to remote infrastructure, platform, or software resources [5].

Cloud computing represents a consistently growing trend in storage of information and access to computational resources, and its use in industry and government, as well as at a customer level, is not likely to decrease in the next few years. A study by Gartner [6] projected 17.2% growth of investments in public cloud services in 2016 compared to the previous year, including an increase of over $208 billion, and an almost 43% increase in investments in IaaS worldwide.

Yet with higher performance and lower costs, the cloud also brings new security vulnerabilities, as the use of shared infrastructures raises the risk of unauthorized access to information, disclosure, and data loss. We are seeing new forms of exploitation of cloud-specific vulnerabilities, such as “side channel” attacks. The presence of multiple tenants on the same infrastructure makes cloud systems behave differently from noncloud systems, and by figuring out this behavior – for example, by analyzing CPU utilization – a malicious attacker residing on the system might be able to infer the information processed in the system by another tenant [7–10]. Similarly, by compromising the hypervisor – the shared platform that governs the functioning of single virtual machines (VMs) – it is possible to compromise the VMs installed on the shared infrastructure.2

Such new vulnerabilities are evolving alongside the known threats and issues that are common to all IT systems; insecure or malicious software, for example, can result in damage regardless of whether there is shared infrastructure. Thus, the need for cloud-specific countermeasures is just one element of the modern cybercrime and security landscape that is forcing governments and industry to pursue new techniques to assure confidentiality, integrity, and availability (C-I-A) of information.3 Among these is the adoption of security standards.

Standards are a mechanism historically used to guarantee the characteristics of products and services and meet users' expectations. If vendors state that their products comply with some particular standard, users can feel assured that the products will possess certain features and meet certain specifications.

By being compliant with a security standard, a CSP shows its commitment to maintaining high levels of security, with an immediate benefit for their clients and a considerable return in building a reputation for trustworthiness. Thus, CSPs are willing to undergo expensive audits conducted by third parties to certify their compliance with certain standards and gain in credibility.

8.1.1 What Is a Standard?

Standards are the result of agreed-upon measures that act as facilitators in everyday life. Standards describe commonly understood concepts, such as the standard time, standards of living, the gold standard, cultural standards, or standards in education.

Through standardization, we achieve “uniformities across time and space… [and we] make things work together over distance or heterogeneous metrics” [12]. In other words, by using a standard we assure a common reference across countries and continents, and consistency over the years in describing the world around us. For example, the use of customary units of volume in the United States allows producers and consumers to have a common reference on the capacity of a bottle of water. It creates the conditions under which producers can sell their bottled water across the nation, and users can buy it knowing exactly the quantity they are paying for.

Standards can be derived from unwritten social norms or can result from negotiation and agreement among parties. More often, in the case of particularly complex or specific matters, standards are promulgated by government or nongovernmental organizations and can be formalized in normative documents with detailed specifications of the requirements to be satisfied. For instance, the Environmental Protection Agency (EPA) specifies the allowed level of contaminants in drinking water, thus setting a standard for those who sell or distribute drinking water in the United States. In such cases, the creation of standards is assigned to experts with a thorough understanding of the issue. As in the example of drinking water, a simple agreement among users and distributors on the characteristics of water is not sufficient to assure crucial qualities, such as safety for human consumption. Experts in the fields of chemistry, biology, and medicine must be involved to determine what chemicals in what quantities should be allowed to assure that contaminants are minimized as needed and that the end result is acceptable to users.

Standards can often act as authoritative rules [12] and be as effective as formal legislative acts or social norms in regulating our society [13]. Regulation through standards can be a task for either governments or nongovernment standards organizations, which are expert groups devoted to creating standards in a multitude of fields. For example, since it was founded in 1947, the International Organization for Standardization has issued more than 21,000 publications [14] in fields ranging from information security to occupational health and safety to food safety management, among many others [14]. Another example is the National Institute of Standards and Technology (NIST), which is part of the U.S. Department of Commerce. NIST is dedicated to measurement and standardization in science and engineering [15].

NIST's definition of standards [16] encompasses two categories that are based on the standards' function. Technical standards clarify the specifics of products or materials and detail the required procedures for managing them, while performance standards define results but do not specify the procedures to obtain them.

An everyday example of a technical standard can be seen in the design of power sockets and plugs. In the United States, since 1917, people can count on using the same type of power socket for the great majority of their home appliances [17], and can expect to move to any other location within the United States and be able to plug in their devices without the use of adaptors. However, Americans traveling to most countries outside the United States will need adaptor plugs in order to use their appliances, and sometimes even the adaptors are insufficient, since differences in voltage make it impossible to use some appliances in some countries. To allow interoperability of power sources and appliances, a technical standard prescribes a specific design for sockets and plugs, as well as a common voltage with which home appliances should work. Producers of electricity-powered devices such as dishwashers and refrigerators need to follow specific requirements prescribed for their products in technical standards; consumers, on the other hand, can expect that the devices they buy are compliant with standards without having to verify the operability of the device in their homes.

An example of performance standards, on the other hand, would be the emission standards that regulate the pollutants produced by engine-powered vehicles. Governments set goals, which are represented by quantitative values for the maximum acceptable emissions of greenhouse gases or other pollutants, but they do not specify what technique or methods should be adopted to achieve these goals. Manufacturers are given the autonomy to decide the best solution to adopt, so long as their vehicles do not exceed the threshold imposed by the government. When purchasing a vehicle, users can expect it to produce emissions below the limit prescribed in the standard, without further verification.

Another distinction among standards, based on NIST's definition [16], depends on their source. Standards are “voluntary” if not imposed by laws or regulations and/or “nongovernmental” if they are not created and enforced by authorities [16]. While a standard can be imposed by government (e.g., the mentioned emission standards), it could instead be adopted on an organization's own initiative, generally following a set of rules suggested by standardization bodies. An example of a voluntary standard is the safety standards used in the food industry. Although not always imposed by law, these standards are adopted to guarantee higher quality of the products and to reassure consumers on the practices used in producing and processing their food. As with technical and performance standards, the purpose of adhering to a voluntary or nongovernmental standard is to induce trust in users and reassure them on the quality of a service or product, avoiding case-by-case verification.

The distinctions explained above illustrate the characteristics and goals of standards. The examples presented thus far hint at how standardization is pervasive in our lives and covers the most disparate aspects of the production and provisioning of services and goods. Information technology and cloud computing, just like any other product, follow the same general principles about standardization. Users' expectations, however, are not limited simply to interoperability or safety and may involve aspects such as confidentiality, integrity, and availability of information processed in IT applications. At the same time, the techniques adopted to meet the requirements of IT security standards are not based solely on either technical measures or performance but rather a combination of both.

8.1.2 Standards and Cloud Computing

To understand the use of standards in cloud computing, we must consider how IT security is deeply affected by the rapidly evolving technologies upon which it builds and the equally rapidly evolving cyberthreats. New technologies often mean new threats and with them the necessity of updated security techniques. Still, users expect a certain level of security in accessing applications and services, and it is the provider's responsibility to meet those expectations.

In this scenario, security standards are an essential tool. By imposing measures and procedures aimed at increasing information assurance, a standard sets a security baseline. By complying with the standard, a CSP can make sure to meet that baseline protection against threats and vulnerabilities. In cloud computing, not only do security and quality of service depend on technical precautions, but careful consideration of the internal procedures adopted by the CSP is also absolutely essential. In spite of the implementation of cutting-edge technical measures, attackers can leverage weaknesses in the information management process to compromise the confidentiality, integrity, and availability of information being managed and processed in the cloud. Technical precautions are necessary and must be balanced with equal attention to the usability of applications while guaranteeing the security of information to avoid unacceptable trade-offs.4 Still, poor-quality internal processes and procedures can negate technical efforts, producing flaws that are easily exploitable by malicious attackers and generating data losses or disclosure.

If a CSP lacks clear policies on the use of workstations or mobile devices, that constitutes a procedural flaw that may open up backdoor access to cloud resources despite the presence of technical security measures. For example, mandatory logouts from individual users' accounts offer an important form of security protection; if employees are not required to log out upon leaving their workstations unattended, their accounts could easily be exploited by other employees to gain access to resources for which the malicious employees might not have the necessary authorization level. Although technical measures may have been implemented, such as least-privilege access rules based on user identity and two-factor authentication to the workstation, a lack of attention to internal procedures can still create a security flaw. (We assume that physical access control procedures would be implemented correctly, limiting the pool of potential attackers to employees who work for the same organization.) For a second example, the absence of password policies or even lock-out screen policies for portable devices (such as smartphones or tablets) assigned to a CSP employee may have even worse consequences. For instance, a mobile device that is accessible off-premises, and can connect to corporate networks and store or access information controlled by the organization, can be targeted by malicious attackers to compromise information controlled by the CSP. A number of different attacks can be performed on mobile devices, for example, enabling remote code execution and privilege escalation (like the so-called “Stagefright” exploit) [18,19], or using multimedia messaging services (MMS) to target Android kernel vulnerabilities [20]. Mobile-specific vulnerabilities can easily be exploited by a malicious attacker if access to the device is not properly protected.

Those two examples suggest the importance of well-designed internal procedures to protect against undesired access and violation of security measures. To assure adequate protection, standards must include security measures encompassing procedures and policies, not just the technical aspects. Compliance with a standard can be instrumental in reassuring users about the security measures implemented by the organization. Furthermore, in a commercial context in which the provider is a private vendor, the ability to show users that the organization complies with security measures can be as important as the implementation of the measures itself. Demonstrating higher-quality products or services than the competitors can make all the difference in the commercial success of a CSP, and users have great interest in verifying that their vendors are implementing strong security measures and best practices.

However, it is not always possible to offer sufficient insights on IT systems and procedures to demonstrate adequate implementation of security measures. In the auditing of a CSP, a number of challenges make it almost impossible for a single user to perform a full assessment. To prevent accidental or malicious damage, for example, access to the CSP's premises must be restricted to a few authorized subjects, and if auditors are admitted, they must be subject to specific rules.

The solution to the impasse is to rely on trusted third parties to perform the audit and assign a “quality label” certifying the CSP's full compliance with the best practices or standards prescribed for the service provider. Standards may refer to a quality or qualification, such as those acquired with a certification [13], and it is in this context that the terms standard and certification become intertwined. Security audits are conducted by professional auditors with specialized expertise in assessing service providers and verifying compliance with the requirements prescribed in a standard. Different standards require dedicated auditors (although auditors are often authorized to perform assessments for multiple standards) and can result in a certification, authorization, or report that can be used to attest compliance with a standard or framework.

8.2 Vision: Using Cloud Technology in Missions

The federal cloud has not been immune to the need to standardize. Following the release of a 25-point strategy for IT management at the federal level, and in response to the need to reduce IT expenditures while maintaining high security standards, the U.S. government created its own assurance standard: the Federal Risk Authorization Management Program (FedRAMP), which was published in 2011. It is based on a set of security controls organized in three baselines: low, moderate, and high.5

FedRAMP prescribes a set of prerequisites that CSPs must meet to offer cloud services to federal agencies and builds on an evaluation and auditing process that culminates in an Authorization to Operate (ATO) for the CSPs. Some agencies, however, have more stringent requirements. That is the case with the Department of Defense (DoD), which – on top of demanding a preliminary FedRAMP authorization – requires further controls that depend on the security impact level of the data transmitted to the cloud provider. The DoD distinguishes six impact levels: Levels 1 and 2 are for unclassified, uncontrolled information; levels 3 and 4 are reserved for unclassified, controlled information; level 5 is for unclassified, controlled information with higher sensitivity than level 4; and level 6 is dedicated information ranging from secret to top secret. For the purposes of authorizing the provisioning of services to the DoD, the Defense Information Systems Agency (DISA) has published a set of guidelines [21] indicating how, once a FedRAMP authorization has been achieved, DISA will perform further verification of additional requirements on the ground before authorizing the CSP. This authorization process (referred to by DISA as FedRAMP+) has one exception, which is the authorization for impact levels 1 and 2; for them, a FedRAMP authorization is sufficient, and no further verification is required in order to obtain a DoD authorization. Since the 2016 publication of a high security baseline for FedRAMP authorization, the FedRAMP governing board and DoD have been evaluating the possibility of automatic access to DoD authorization up to impact level 4 for CSPs with a FedRAMP high ATO [22]. If we consider the importance of unclassified, controlled information, it is easy to see how wrong calibration of the measures imposed in FedRAMP can result in serious consequences for the agency relying on the authorization process.

Information covered by assurance level 4 includes, but is not limited to, personally identifiable information (PII), including Social Security numbers and medical records of employees; information regarding ongoing investigations; and information disclosing infrastructure vulnerabilities, design, or construction information [23]. Loss or disclosure of such controlled unclassified information may have serious effects, so it must be protected adequately.

The purpose of FedRAMP is to set a baseline of security requirements for (i.e., to standardize) cloud services for the federal government. It benefits federal agencies by streamlining the authorization process so that the agencies are not forced to do case-by-case evaluations of the security measures implemented by CSPs. By referring to a standardized authorization assessment and procedure, federal agencies can save time and resources by requesting a product or service that is compliant with well-known and verifiable requirements.

8.3 State of the Art

In the United States, FedRAMP was created to build trust in cloud security following a period of increased attention to cloud computing from the federal government. To reduce high expenditures on IT services at a federal level,6 at the end of 2010, the Office of Management and Budget (OMB) released the “25 Point Implementation Plan to Reform Federal Information Technology Management” [24]. The Implementation Plan was followed a few months later by the “Cloud Computing Strategy” [5], which set a roadmap for migrating federal IT services to cloud environments. The first release of FedRAMP was at the end of 2011; its structure is similar to that of the existing Federal Information Security Management Act (FISMA), with which all federal IT systems must comply, including on-premises infrastructures. Upon its release, FedRAMP joined other existing frameworks for IT security auditing, assessment, and certification requirements.

Two of the most commonly used standards in the IT industry were designed to demonstrate strong protection of privacy and security in IT systems; they include Service Organization Control (SOC) audits, which are based on the Trust Services Principles and Criteria (TSPC) developed by the American Institute of Certified Public Accountants (AICPA), and the ISO/IEC 27001 certification, which was built on criteria published by the International Organization for Standardization (ISO) jointly with the International Electrotechnical Commission (IEC).

ISO/IEC certifications represent a well-established international benchmark for a variety of standards in different fields. One of the prevalent frameworks for the regulation of IT best practices is the ISO/IEC 27001. This standard has regularly evolved to keep pace with technical innovation by prescribing security techniques for information security management systems. Another set of standards is similar to ISO/IEC certifications and FedRAMP, as it is used to attest the security of IT systems: the AICPA SOC reports.

Although the conclusions of AICPA's SOC reports cannot technically be defined as “certifications,” they share some of the characteristics and goals of the certifications and authorization standards that were highlighted in previous sections of this chapter. The specific controls performed in SOC audits may vary, touching upon slightly different things in different situations. Thus, the result of an independent assessment is more properly defined as a “report,” as it is based on a selection of controls determined on a case-by-case basis. Even so, assessment criteria are both well-defined and mandatory in SOC reports, and that can be used to increase the confidence of customers or stakeholders in the security measures implemented by an organization [25] and reassure them that the service will meet their expectations. To that extent, we will consider SOC reports alongside other frameworks, such as FedRAMP and ISO/IEC 27001, that share the same underlying purpose, and we will group them together in the category of standards and certifications. Although they do not define in detail the controls to adopt, but rather establish guidelines, the technical and procedural criteria that are used for SOC reports must be used to assess the service organization and provide auditors with detailed guidance, as the criteria specify not only a goal (the security of information) but also the procedure or methods to achieve it. Among the three possible SOC reports, the most specific and complete is SOC 2, which has the goal of “Reporting on Controls at a Service Organization Relevant to Security, Availability, Processing Integrity, Confidentiality or Privacy” [25] and is conducted by an independent auditor who will attempt to verify that the organization meets the criteria of the TSPC.

In TSPC, and consequently in SOC 2 reports, requirements and controls are thorough and appear to be comparable to the ISO/IEC 27001 provisions and NIST provisions for FedRAMP. Although the similarities of the three standards can easily be seen, no formal effort has been made to enable mutual recognition among them.

8.3.1 The Federal Risk Authorization Management Program

The effort to bring FedRAMP into being started in 2009 as a result of the Federal Cloud Computing Initiative [2,26] and the cooperation of multiple agencies,7 local governments, academia, and nongovernmental organizations [27], which worked together to create a standardized approach for assessment and authorization procedures for providers of cloud services to the U.S. government [27]. The FedRAMP effort not only created a security framework for cloud services in compliance with FISMA [28], but it also moved to a “do once, use many times” approach, which allows federal agencies to leverage existing authorization processes instead of creating a new assessment procedure every time an assessment is needed; thus, it helps reduce federal agencies' IT expenditures in alignment with the “25 Point Implementation Plan” [24]. FedRAMP was officially initiated in December 2011 with the publication of the OMB's “Security Authorization of Information Systems in Cloud Computing Environments: Memorandum for Chief Information Officers” [27], and it released its first authorization in May 2013.8

FedRAMP assessment follows the Risk Management Framework suggested in NIST 800-37 [30]. This framework specifies that assessments should be conducted in four steps: (i) In the initial step, the auditor collects information on the system; (ii) then, the auditor performs an assessment of security requirements; (iii) next is the authorization phase, at which point the CSP is allowed to supply its services to the designated agency; and (iv) finally, there is a continuous monitoring phase, intended to guarantee that the same level of security will be maintained after the initial assessment has been performed.

The FedRAMP authorization process offers to CSPs three different paths to obtaining a FedRAMP Authorization to Operate (ATO), and it makes it possible for multiple agencies to leverage the same authorization. For all three options, the CSP is required to undergo an assessment procedure and submit the results of the assessment to the FedRAMP Program Management Office (PMO), which will add the authorized system to the list of compliant systems [28]. The assessment must be conducted by an independent auditor that is among those listed as FedRAMP Third-Party Assessment Organizations (3PAO). Organizations wishing to become 3PAOs must themselves be certified by the American Association for Laboratory Accreditation against the ISO/IEC 17020:2012 requirements9 [32]. The independent auditor performs the necessary assessment of the CSP in terms of the FedRAMP controls, attesting to the CSP's compliance. In the first possible path to obtaining an ATO, the FedRAMP Joint Authorization Board (JAB) and the CSP work together directly. At the end of the assessment, the CSP will be considered “FedRAMP ready,” meaning that all the requirements are satisfied and federal agencies can issue their own ATOs with no further assessments or reviews. In the second option, the CSP works on its own initiative to comply with FedRAMP in anticipation of future authorization. The results of the assessment are submitted to the FedRAMP PMO, which reviews the results and marks the CSP as “FedRAMP ready.” The third possible path is initiated at the request of an agency issuing the ATO, which works together with the CSP to comply with the FedRAMP requirements. Also in this case, after the 3PAO's assessment of the CSP is complete, the request for the ATO must be submitted to the FedRAMP PMO [28].

Following the same approach adopted in FISMA, the FedRAMP assessment process includes a selection of controls derived from NIST Special Publication (SP) 800-53. This Special Publication is part of the 800 series, which “reports on ITL's research, guidelines, and outreach efforts in information system security, and its collaborative activities with industry, government, and academic organizations” [33]. SP 800-53 includes technical and procedural controls aimed at protecting privacy and security in federal information systems and organizations. It is organized into 18 control families, each of which is made of controls and control enhancements. The goal of each control is to provide specific guidance to service organizations on increasing privacy and security, while the enhancements prescribe further measures for improving the protection assured by the control from which they descend. SP 800-53 is structured according to three distinct baselines, which were chosen based on the impact levels described in Federal Information Processing Standard (FIPS) 199 and the security control selection process specified in FIPS 200. The three baselines are a low baseline, which includes controls for systems that process information for which a loss of C-I-A would have limited impact; a moderate baseline, which relates to information for which a loss of C-I-A may result in serious damage to the organization; and a high baseline, which covers information for which a loss of C-I-A could have severe or catastrophic consequences for the organization and its activities [34]. FedRAMP follows an identical organization and builds on the same three impact levels.

In 2011, the year of its first release, FedRAMP considered only low and moderate security baselines for a total, respectively, of 116 and 297 controls and enhancements divided into 17 families, as selected from the 810 controls and control enhancements detailed in NIST SP 800-53. In 2014, following a new review of NIST SP 800-53, FedRAMP was updated to include more controls and enhancements, reaching a total of 325 for the moderate baseline and 125 for the low baseline. A high baseline was added to the FedRAMP program in July 2016, bringing an additional 96 controls and enhancements. The introduction of the high baseline allows federal agencies to move to cloud systems 20% of federal information that by itself accounts for 50% of the total federal spending on IT [35].

As of May 2017, 81 products have been authorized, 4 at a high level; 62 products are in the process of being approved by the FedRAMP PMO, and another 10 are ready for authorization, although they have not yet been authorized by any agency. The total number of 3PAOs is 43, but only 18 of them have performed at least 1 FedRAMP assessment in the 6 years of the program [36].

8.3.2 SOC Reports and TSPC

The American Institute of Certified Public Accountants (AICPA) has worked since 188710 on activities that include, but are not limited to, advocacy, certification, standardization, communication, and education in the accounting profession [37]. AICPA counts more than 418,000 members, most of whom are involved in public and management accounting (representing 47 and 34% of the voting members, respectively), and is active in 143 countries worldwide [37].

One of AICPA's standards, the Statement on Auditing Standards (SAS) No. 70, was released in 1992 to support examination of organizations' financial statements and was widely adopted by service organizations to demonstrate compliance with security and privacy best practices [38]. A SAS 70 audit consisted of an assessment by external auditors of a service organization's procedures, policies, and security controls, and concluded with reports that documented observations on possible security shortcomings and subsequent improvements [39].

SAS 70, however, was conceived to achieve compliance with financial regulations such as the Sarbanes–Oxley Act in the United States, and its adoption as an attestation of compliance with security and privacy best practices was essentially a misuse. Furthermore, SAS 70 reports were conceived as an auditor (on the financial statements of the service organization) to auditor (on the financial statements of the user organization) document but evolved over the years to include a marketing element, as the reports were made available to prospective clients to demonstrate the use of security controls in the organization [39]. The inclusion of the marketing element led AICPA to revisit the standard as part of the Clarity Project conducted in 2011 by the Auditing Standards Board, which created distinctions between different reports according to the purposes they served, and created the Service Organizations Controls (SOC) reports [40].

Audits are carried out by independent third parties and produce three main types of SOC reports: SOC 1, SOC 2, and SOC 3.

SOC 1 reports serve the original purpose of SAS 70 reports, as they address controls that are relevant for the audited entity's financial statements and can be differentiated into Type I reports, which address the suitability of the design of the controls chosen to perform the assessment according to the control objectives set by the management, and Type II reports, which examine the suitability of the design and operating effectiveness of those controls. The main difference between the two types is the duration of time covered: A Type I report is limited to a specific moment in time and attests that an organization has selected a number of controls to be adopted, while Type II reports attest implementation of those controls over a longer period of time.

While SOC 1 is focused on financial statements, the SOC 2 and SOC 3 reporting standards attest to the security and privacy measures utilized in the service organization. SOC 2 and SOC 3 reports are released following an assessment that verifies compliance with the list of Trust Services Principles and Criteria (TSPC) published by AICPA [25], which were designed to help organizations promote the C-I-A principles. SOC 2 reports are thorough and are available as either Type I or Type II, with the same distinction and qualities used for SOC 1 reports. Although SOC 2 reports are used to reassure customers and stakeholders on the implementation of security and privacy measures within an organization, they are not public but can be made available upon request. SOC 3 reports are similar in scope to SOC 2 reports, but consist of brief statements rather than complete analyses of security measures' compliance with the TSPC; they are generally available to the public, and thus are suitable for the marketing applications that (improperly) characterized SAS 70 until it was repealed in 2011 [25]. To issue a SOC report, auditors must be certified by AICPA as Certified Public Accountants (CPAs).

The first version of the TSPC that is relevant to the SOC reports was released in 2009; thorough revisions were done in 2014 and 2016.11 The TSPC documents are structured in multiple sections, of which section TSP 100 specifies “Trust Services Principles and Criteria for Security, Availability, Processing Integrity, Confidentiality, and Privacy.” That section is used as the basis for the SOC 2 and SOC 3 assessments and reports.

TSPC 2009 presents 117 criteria organized into four groups based on the principles that they protect: confidentiality, integrity, availability, and the more general principle of security. Within each group, the criteria are split into four categories: (i) policies, which oversee the creation of relevant documentation to protect the mentioned principles; (ii) communications, which aim to make sure that policies are shared and approved across the organization; (iii) procedures, which establish formal mechanisms for implementing the necessary measures to help the organization meet its security, availability, integrity, and confidentiality objectives; and (iv) monitoring, which relates to enforcement of measures, policies, and procedures across all levels of the organization. Privacy criteria are not included in the TSPC, which for their definition refer to the more extensive list of Generally Accepted Privacy Principles (GAPP). GAPP was created in 2009 as a collaborative effort of AICPA and the Canadian Institute of Chartered Accountants (CICA).

The TSPC were heavily revised in 2014; the content was radically reorganized into 4 groups and 47 criteria. The great reduction in the number of criteria reflects the creation of a new group made of common criteria that were applicable to all of the C-I-A principles, thus reducing repetition in the document and simplifying assessment activities for organizations and auditors. The remaining content was reorganized into three other groups that contained additional criteria specific to each of the C-I-A principles. Like the 2009 release, the 2014 version of TSPC refers to GAPP for privacy criteria.

In 2016, further changes were made to the TSPC criteria. General improvements were made to the definition of the scope of the criteria, and some syntactical changes optimized the content and clarity, but the most relevant innovation in the 2016 issue was the introduction of additional privacy controls in a dedicated group. TSPC 2016 includes 44 general criteria (counting both those that are common to all principles and those that are C-I-A-specific) and 20 additional privacy criteria, all organized in 5 groups. The privacy-specific section introduced with TSPC 2016 supersedes GAPP.

8.3.3 ISO/IEC 27001

ISO/IEC 27000 is a large family of standards that is the result of a joint effort between the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). In 1987, ISO and IEC together created the Joint Technical Committee 1 (JTC1) to cover the field of “Information Technology.” Later, in 1989, they collaborated on the Subcommittee 27, which developed 151 published standards specifying security techniques in IT environments [41]. Among these standards, The ISO/IEC 27001 Information Security Management standard prescribes requirements and procedures for the implementation, maintenance, and improvement of Information Security Management Systems (ISMSes). The standard addresses measures that are common to all ISMSes and does not differentiate among different types of organization undergoing the certification process [42].

Today, ISO/IEC 27001 is the basis of more than 27,000 certifications across five continents worldwide; indeed, there has been a consistent increase in its adoption. For example, there was a 20% growth in ISO/IEC 27001 certifications in 2015 compared to the previous year, with the North American region registering the highest increase, 78%. In absolute numbers, the highest concentration of certifications is in East Asia and the Pacific, followed by Europe, Central and South Asia, and North America [43]. (Together, East Asia, the Pacific, and Europe account for more than 22,000 certifications.)

The original precursor of ISO/IEC 27001 was called British Standard 7799, which was created in 1995 and revised in 1998 to become ISO 17799. ISO and IEC worked together in 2005 to convert the ISO 17799 into ISO/IEC 27001 and ISO/IEC 27002 [44]. ISO/IEC 27001 is organized in general clauses that provide guidance to auditors and organizations, plus more specific controls to be implemented to achieve compliance with the standard. The controls were derived from those in ISO/IEC 27002, which was created at the same time as ISO/IEC 27001, and were selected for their relevance in the creation and maintenance of an ISMS.

The first release of ISO/IEC 27001, in 2005, included 133 controls in 11 control groups. The standard was revised in 2013 to provide enhanced flexibility and interoperability with other standards and to update the standard to keep pace with the evolution of technology, including protections for new challenges such as identity theft and mobile security [45]. After that review, the number of controls was reduced to 114 across 18 categories. In ISO/IEC 27001:2013, the P-D-C-A (Plan, Do, Check, Act) approach is still considered a best practice, and thus is regularly adopted despite its no longer being required by any explicit provision [46]. (The 2005 version considered the approach mandatory.) The first step for the organization undergoing the certification process, Plan, requires consideration of goals and objectives and careful planning of the controls to be enacted; the second step, Do, draws from the controls and security measures identified in the first phase, to proceed to their implementation; the third phase, Check, is aimed at overseeing the controls implemented in step two, evaluating their effectiveness; and the fourth and last phase, Act, works on the flaws identified in the previous step, making further adjustments where required. The organization obtains a certification after the successful completion of the P-D-C-A cycle by an accredited auditor who is following the criteria and controls in ISO/IEC 27001; periodic controls and independent audits are required to maintain the certified status.

Auditing organizations that perform ISO/IEC 27001 assessments must be certified against both ISO/IEC 1702112 and ISO/IEC 27006.

8.3.4 Main Differences among the Standards

The three standards, ISO/IEC 27001, SOC, and FedRAMP, have important similarities, including similar goals. Security and privacy are among the principal objectives of all three standards, and all three are structured on a given number of criteria and controls that are to be assessed by independent and accredited third parties. Yet some differences among them can be identified, giving a more accurate idea of their range and scope.

ISO/IEC 27001, SOC, and FedRAMP are all grounded in careful evaluation of management processes and targeted implementation of controls to guarantee the integrity and security of information [44]. All of them examine infrastructure, internal procedures, and risk management. FedRAMP, however, focuses on cloud services offered by private outsourcers to the federal government. Although the most recent versions of ISO and SOC include elements useful for cloud security assessments, they have a more general scope and require service providers to guarantee the C-I-A principles regardless of what type of service they offer; thus, their security assessment tools are available for use by any kind of ISMS or service organization. The difference in scope is connected to the type of guidance and the number of controls required in the three different standards. Further, FedRAMP requires at least twice as many controls as the other standards, reflecting the fact that FedRAMP's controls are more specific and detailed, while those in ISO/IEC 27001 and SOC/TSPC are more general. SOC 2 and ISO/IEC 27001 do not specify how to implement the criteria and controls that they require, showing higher flexibility than FedRAMP.

Perhaps the most notable differences are in geographical distribution and diffusion. FedRAMP is limited to U.S. federal agencies and enforced by U.S. auditors; ISO/IEC 27001 and SOC are international standards with global applicability and auditability. Furthermore, the United States – with its 1247 total certificates, according to the most recent data available (from 2015) [43] – is among the 5 countries with the highest numbers of ISO/IEC 27001 certificates.13 It has far more ISO/IEC certifications than FedRAMP authorizations, the count of which stood at 348 authorizations among 81 providers as of May 2017. In some cases, the same CSP has received multiple ATOs, each issued for a different product [36]. Because TSPC and SOC do not result in formal certifications, but rather in personalized reports, no current data are available on their worldwide implementation.

Finally, the times of the three standards' releases and the gaps between their updates show visible divergence. Whereas ISO/IEC's only review of the 27001 standard was in 2013 (8 years after its first publication), SOC and FedRAMP have received more frequent updates (after 2 and 5 years and after 4 years, respectively).

8.3.5 Other Existing Frameworks

The landscape of security standards tailored or adaptable for cloud computing systems does not consist only of FedRAMP, ISO/IEC 27001, and SOC 2. The other existing frameworks, however, are of less interest for our purposes. Some are data-specific, meaning that they focus on particular qualities of the target system that are related to the type of data processed by the system, such as financial data. Some are not supported by external audits that prove compliance with the provisions in the standard. In addition, the popularity of some of them is not significant compared to that of the three standards analyzed in this chapter. Even so, in order to provide a more comprehensive overview of the IT security certification landscape and note some similarities in other frameworks, in the following section we briefly note three other standards that have attracted some interest for cloud computing. PCI-DSS

The Payment Card Industry (PCI) Security Standard Council was founded in 2006 to help anyone involved with credit card payments, such as merchants or financial institutions, with the implementation of standards for security policies and technology [47]. The PCI Data Security Standard (PCI-DSS) creates a structure for developing a payment card data security process [47]. PCI-DSS is built on a three-step process (Assess, Remediate, Report) and includes specific controls and directions on how to implement these controls to cover security procedures, access control, and software development [48,49]. Organizations that process cardholder data are required to comply with the standard, and their compliance must be attested by third-party Qualified Security Assessors (QSA) authorized by PCI. C5

The Bundesamt für Sicherheit in der Informationstechnik (BSI–German Information Security Office) presented its Cloud Computing Compliance Control Catalogue (C5) in February 2016. The C5 is a set of cloud-specific security and privacy controls, largely based on ISO/IEC 27001 and the SOC reporting standard [13,50]. C5 was intended merely as a guideline for CSPs, and as a supplement to other certifications. However, C5 can also be used as a standalone framework to verify the adoption of technical and procedural measures promoting cloud assurance. Like other frameworks, C5 relies on security controls organized into control domains. It prescribes a set of basic requirements that clouds are required to meet to be considered compliant. Additional requirements are specified to assure higher security levels, where applicable. STAR

The Cloud Security Alliance (CSA) is a nongovernmental organization whose core mission is to develop best practices for cloud assurance and to promote awareness of best practices in cloud environments [51]. Its activities include designing security principles and attesting to their implementation in cloud systems. In 2013, CSA released a security assurance program known as the Security, Trust & Assurance Registry (STAR). The goal of STAR is to offer a cloud-specific certification built on the principles outlined in ISO/IEC 27001, with enhanced controls from other existing schemes (e.g., FedRAMP and SOC 2). Although STAR has not attained the popularity of other standards – only 44 certifications had been issued as of July 2016, it has attracted interest as a self-assessment framework; as of July 2016, 187 self-assessments have been performed [52]. STAR's assurance framework is built on CSA's Cloud Control Matrix (CCM), which is a list of 133 controls and criteria organized into 16 “families” that encompass both technical and procedural security measures. The CCM was first released in 2010 and has been regularly updated over the years; it was last revised in June 2016 [53].

8.3.6 What Protections Do Standards Offer against Vulnerabilities in the Cloud?

Standards set a baseline for protection against vulnerabilities in the cloud and aim to reassure users about the privacy protections and security offered by the service that processes their data. Users' main concerns are the common threats, issues, and vulnerabilities that affect cloud environments. Thus, the adequacy of a standard is directly connected to the protections its proposed measures offer against those issues and vulnerabilities. Depending on the type of issue that could affect the cloud and the type of information processed in the system, the potential consequences for the C-I-A of information processed in the cloud may vary, as may the need for controls that protect against some threats [34]. For example, denial-of-service (DoS) attacks, which are often used as “red herrings” to conceal other forms of attack [54] and can cause serious inconvenience to users and even heavy costs for businesses, can compromise the availability of information for only a limited amount of time. On the other side of the spectrum, side-channel attacks do not have significant consequences for the availability and integrity of information, but they do compromise confidentiality. An ideal security framework should consider those trade-offs to assure higher protection against vulnerabilities when it is more necessary, while optimizing resources and effort. Thus, to maximize the value of both an assessment and the implemented security measures, when the initial objectives of an assessment are being set, the mission of the organization should be identified and the necessary controls should be selected according to the threats, issues, or vulnerabilities of greatest importance in the specific scenario [55]. That approach is a good fit for IT security assessments of organizations that manage their processes and infrastructure on their own premises. However, it might be less effective for assessing vendors of cloud services, as the services of the CSP may be accessible to a wide range of organizations and users (e.g., in health care, the financial sector, or manufacturing) that have different security needs. Industry-based certifications exist and are often built on requirements imposed by sectoral laws (e.g., SOC 1 aims to fulfill the requirements of the Sarbanes–Oxley Act). The broad context in which CSPs operate, however, and their consequent need to comply with multiple standards simultaneously, calls into question the desirability of having multiple concurrent standards.14 Instead, it suggests the need for a more comprehensive and complete framework that is equally aware of the confidentiality, integrity, and availability of information.

It is necessary to ensure the completeness of a standard, including its ability to protect against issues and vulnerabilities; but that may not be enough. The large variety of vulnerabilities in cloud computing, including both cloud-specific issues and issues independent of the shared infrastructure, must be covered. At the same time, the study of new vulnerabilities is constantly advancing, and work in academia and industry as well as nongovernmental organizations is contributing to the identification of potential threats, the building of awareness, and the improvement of countermeasures. Thus, there is tension between the need to standardize protection methods and controls and the constant discovery of new issues that must be included in the standardized protections.

Thus, the study of vulnerabilities must be closely connected to the maintenance of security standards in order to keep the standards effective over time. It is necessary to determine how a standard reacts to new threats, whether the measures and controls that it imposes are adequate and effective, and whether the baseline protection is maintained, despite the ever-changing threat landscape. Resilience is thus a fundamental quality of security standards in cloud computing systems, especially in light of the unpredictability of vulnerabilities generated by new threats. That connection, however, is not prominent in the literature, and studies of new threats and issues conducted in academia and industry typically address threats on a case-by-case basis [58].

The work of the CSA represents a partial exception to that trend. It has been conducting an ongoing study of cloud issues since 2010, and it regularly updates its published list of issues in cloud computing. Its first major publication, “Top Threats in Cloud Computing” [59], was followed by “The Notorious Nine: Cloud Computing Top Threats in 2013” [60], and then “The Treacherous Twelve: Cloud Computing Top Threats in 2016” [54]. CSA's studies are based on surveys of experts who are asked to identify and rank in order of severity the most common issues in cloud computing, such as data breaches, advanced persistent threats (APTs), malicious insiders, or account hijacking. What differentiates CSA's work is that although it is aimed primarily at providing guidance to cloud providers and other stakeholders in evaluating the adequacy of their systems, the studies have direct references to CSA's CCM, which serves as the basis for the STAR certification. Thus, there is a direct connection between the controls in the Matrix (i.e., the standard) and the issues highlighted in the study. In their most recent publication (from February 2016) [54], the Cloud Security Alliance featured 12 issues in cloud computing, for which it gave direct reference to the relevant controls in the CCM. Out of the 133 controls in the Matrix, 82 were noted as offering protection against 1 or more of the “Treacherous Twelve” issues, while the remaining 51 controls do not have a direct connection with any of them.

The work of the CSA is at the foundation of our study, since it helps make a connection between the three security standards analyzed in this chapter, and the common cloud-specific threats and issues. In the following sections, we consider the effectiveness of AICPA SOC 2 (which refers to TSPC), ISO/IEC 27001, and FedRAMP, and compare their performance with the requirements imposed by a third-party framework: the CSA CCM v.3.0.1. We assess the number of controls in the CCM satisfied by each of the four standards and evaluate the relevance of the missing controls by assessing the severity of the threats with which they cannot cope.

8.4 Comparison among Standards

The existence of multiple standards with similar characteristics, all covering security in IT environments (including the cloud), raises a number of questions on their necessity and function. Are there benefits of having multiple frameworks, when the final goal is to achieve the best possible security – or at least fulfill a baseline – and reassure users on the quality of the service that is being offered?

The adequacy of each standard in addressing current threats is another point that requires attention. How does a standard perform in the face of the current, dynamic threat landscape? Are existing standards resilient to the appearance of new issues and vulnerabilities? Can we observe any trends in the protection they offer that can be used to improve their adequacy and resilience?

Standards can be different, as they are often framed around divergent principles and strategies to assure confidentiality, integrity, and availability of information. A standard can be more oriented toward documentation of processes and internal procedures in information management, or have a stronger emphasis on the adoption of technical measures. But with regard to how good a standard is at fulfilling the C-I-A principles, it is necessary to evaluate the standard's ability to offer baseline protection against issues and vulnerabilities. A differentiation in principles can be drawn and different solutions adopted, but the process integrity or the security of applications must always be the final goal.

For example, two standards can approach the same problem from two different perspectives. In regulating physical access control, one standard might be broad and demand that an organization issue control policies that follow a least-privilege rule that requires users to authenticate whenever they transition between physical environments, such as moving from a service room to a data center. Another standard could prescribe the use of strong (two-factor) authentication to verify identity upon physical entry of premises but not mention what the control policy should be or how frequent controls should be (e.g., only upon the first access to the premises or upon any transition between two environments).

In that example, there are two possible flaws: one derives from insufficient technical measures, and the other from lack of policies. If strict policies are implemented, but no attention is given to the technical means for authentication of visitors, the general level of security upon the first access to the premises, as well as in the transition between environments, will be low. Thus, once the authentication system has been compromised, the use of strong policies to separate the environments and define access levels will become useless. On the other hand, if authentication methods are strong enough, it is harder to compromise the system, thus making accesses to the premises more secure. However, a lack of clear access privilege policies may allow uncontrolled transitions between environments and unauthorized accesses to resources. For instance, persons authorized to enter a service area (e.g., air conditioning maintenance personnel) could easily access the data center if the environments are not segregated and access control is not implemented at every level.

To be complete and adequate to address threats and vulnerabilities, standards could use a technical or policy-oriented approach, but either way they must compensate for possible flaws caused by the specific approach they choose. To form a clear vision of such considerations, we can compare standards to each other to highlight their differences and weaknesses. However, a direct comparison, in which we gauge how the controls in one standard match with the provisions of another, might lack objectivity and therefore do a poor job of addressing completeness and adequacy.

Direct comparisons of AICPA TSPC (which serves as the basis of SOC 2), NIST SP 800-53 (the basis for FedRAMP), and ISO/IEC 27001 exist, but only a very limited number of studies compare more than two standards to each other [61–63]. NIST, for instance, released in its last version of the SP 800-53 a direct comparison with ISO/IEC 27001. That comparison limits its observations to the annex in ISO, excluding the introductory clauses that give further specification and thus limiting the completeness of the comparison. Furthermore, the table in NIST SP 800-53 is merely an analysis of how the two sets of controls can match each other, regardless of the threats or vulnerabilities addressed by each control or its effectiveness. Similar comparisons between the TSPC and NIST SP 800-53, and between TSPC and ISO/IEC 27001, have been published by AICPA.

To assess the effectiveness and completeness of the three standards, a more nuanced analysis is required. First of all, they must be observed for their adequacy in addressing cloud issues independent of the adopted approach, which could have a technical emphasis or be more oriented toward the completeness of policies and procedures. Second, the evaluation cannot be built on a reciprocal comparison that uses one of the standards as a reference and observes how the others satisfy its requirements. Third, the observations cannot be limited to the latest version of each standard but must include consideration of how the standard has adapted to the threat environment.

8.4.1 Strategy for Comparing Standards

To analyze the differences among the security and privacy measures in FedRAMP, ISO/IEC 27001, and SOC 2, and to perform a study of them over time, we compare the provisions specified in the clauses and controls upon which the standards are based in their available versions, which include NIST SP 800-53 in its 2012 and 2015 releases; AICPA TSPC 2009, 2014, and 2016; and ISO/IEC 27001:2005 and 2013. Since FedRAMP is based on three possible security baselines (low, moderate, and high), we chose to look at the highest baseline that is common to the two releases of the standard, namely, the moderate baseline, and to look only at the controls in the NIST publication that are relevant for FedRAMP authorization. We also include all the possible AICPA TSPC controls that cover security, availability, processing integrity, confidentiality, and privacy, and limit our study to the three versions used for SOC 2 and SOC 3 reports.15

For the comparison, we chose to use a third-party framework that was specifically created for cloud computing, has been updated to encompass protection against newer threats, and is organized in a structure similar to that of FedRAMP, ISO/IEC 27001, and TSPC. The CSA Cloud Control Matrix version 3.0.1 satisfies all those requirements, as it is built on cloud-specific security and privacy controls and is constantly being updated to provide protection against issues and vulnerabilities that target cloud environments. In the following, as we match the controls and requirements of the three standards to the CCM, we will highlight the controls in the CCM that are covered by the three standards, and others that are not sufficiently covered by the three standards. To complete this task, we will rely on the matching made by CSA between the CCM and TSPC 2009 and 2014, FedRAMP rev. 3, and ISO/IEC 27001 [53]; we will partially integrate the matching presented by CSA for FedRAMP rev. 4 [64]; and we will provide the entire matching for TSPC 2016 by building on the same criteria followed by CSA for the matching of its previous versions.

In addition, we will take advantage of CSA's studies of issues, threats, and vulnerabilities in cloud environments and their direct reference to the CCM to select the controls that have a direct connection to any of the “Treacherous Twelve” issues identified in CSA's latest study, from 2016 [54]. As a result of our selection and the matching of the three standards to the CCM, we can verify the impact of the standards on current issues in cloud environments.

8.4.2 Patterns, Anomalies, and Discoveries

The three standards were introduced over a span of 11 years, starting in 2005 with ISO/IEC 27001. Despite being outdated by the appearance of new threats and vulnerabilities, the 2005 ISO/IEC 27001 standard is similar to the other frameworks in terms of its adequacy and comprehensiveness (see Figure 8.1). Its newer version, however, stands out for completeness; the number of matches increased from 91 in 2005 to 130 in 2013. In general, however, completeness does not correlate with the newness of the standard. The completeness of the TSPC-based SOC 2, for example, shows a fluctuation over the 7 years from 2009 (the year of its first release) to 2016; the number of matched controls went from 93 in 2009 to 85 in 2014, and up to 91 in 2016. Finally, FedRAMP showed significant improvement, going from 88 matches in the 2012 version to 103 in the version released in 2015.

Figure 8.1 Matching controls in each standard over time.

The great improvement in ISO/IEC 27001 is undoubtedly due to the 2013 introduction of controls addressing mobile security, which was not considered in 2005. In the CCM, the control family that includes controls for mobile security contains 20 controls, which specify policies and technical measures. For example, CCM requires “Anti-malware awareness training, specific to mobile devices” for the employees of the CSP, or clear policies defining limits and protection in bring-your-own-device use for the CSP's employees. At the same time, more technical measures include the adoption of automatic lockout screens to protect devices from unwanted access [53]. None of these controls were included in the older version of ISO/IEC 27001. The area of mobile security represents a major divide between ISO/IEC 27001 and all the other standards in any of the versions we observed. FedRAMP rev. 4 shows slight improvement in this area relative to its older version; the introduction of more specific measures and controls for mobile devices16 has improved its coverage, reducing the number of omitted controls from 20 to 14.

Similarly, the entire control family that protects the interoperability and portability of cloud applications and data is fully covered in ISO/IEC 27001:2013, but missing from all the other standards in our analysis. This control family consists of five controls aimed at facilitating migration of applications and data and access to information by the tenant through the use of open standards and mutually agreed-upon procedures.

The absence of this control area, however, must be evaluated with respect to the function of each particular standard and its goals. In the case of FedRAMP, which is required only for CSPs that provide services to federal agencies, the applicability of federal laws affects the impact of any omission. Since the U.S. government and NIST have addressed the issue of portability and interoperability in a separate document, NIST SP 500-293, the omission of this control in FedRAMP is not particularly startling.

For TSPC, the introduction of a more homogeneous set of privacy criteria and a slight reorganization of the content in 2016 clearly improved its effectiveness and completeness in comparison with the preceding version. The same cannot be said for the drastic reorganization done in 2014, which worsened the performance, that is, increased the number of omitted controls.

The scenario changes if we introduce another variable into our analysis, namely, the impact of the omitted controls with respect to the current issues and vulnerabilities in cloud computing environments. When we select only the controls in the CCM that are connected to at least one of the “Treacherous Twelve,” we notice a considerable increase in the percentage of controls of the considered standards that match the CCM, counted on a total of 82 relevant controls (see Figure 8.2). The average percentage increase is close to 14%, with the greatest improvement being for FedRAMP rev. 3, for which the number of matching controls moved from 66 to 85%. ISO/IEC 27001:2013 is the only standard showing a slight decrease of 1%.

Figure 8.2 Total percentages of matching controls and a selection of 12 “Treacherous” controls.

Interestingly, except for the TSPC, the results of the selection show a proportionate increase of matching controls for all the frameworks between their older and newer versions. Not only does TSPC 2014 match fewer controls in the CCM than did TSPC 2009, but even the improved TSPC 2016 still has fewer matches than TSPC 2009. The very similar structure of the two most recent versions could be to blame; except for a few adjustments and the addition of the privacy section, the criteria were largely unchanged from 2014 to 2016. TSPC 2009, which was more specific for each principle of the C-I-A, gave more precise directions on the nature of the controls to be implemented. However, we must keep in mind that TSPC is a set of general criteria, and that the SOC reports are based on controls that are built on those criteria but chosen by the CSP and assessed by a CPA. While that may explain the fluctuation and indeed provide partial justification for the omissions, it is a reminder of the importance of choosing adequate controls to fulfill the criteria when they do not fully specify the security measures to be implemented.

A final observation that emerged from the selection of controls concerns the overlap of the three standards. No one control was omitted by all three standards, and that result does not change if we restrict the analysis to the most recent version of each standard. While the presence of ISO/IEC 27001:2013 in the analysis obviously reduces the possibility of overlapping omissions (since only two controls are omitted in ISO), even if we narrow the observation to just TSPC and FedRAMP, we find that only two controls are missing in both. Those two controls are both from the control family that covers virtualization security, and are poorly addressed in TSPC and FedRAMP because they do not clearly require security measures for overseeing the use of virtualization-aware assessment tools or the hardening of operating systems in the virtualization stack.

8.5 The Future

The observation of omitted controls that emerged from our comparison of TSPC, FedRAMP, and ISO/IEC 27001 against CSA CCM answers our questions on the effectiveness and completeness of each standard. Although gaps and weaknesses emerged from the analysis, a more careful observation reveals that they offer a good security baseline when confronted with current issues, threats, and vulnerabilities. We also observed that the three standards show high complementarity and can compensate for each other's omissions. That answers the question of what benefit can be obtained by complying with multiple standards at the same time. Multiple certifications assure more complete protection against issues and vulnerabilities, since each one's focus helps cover for contexts in which the others are less effective. However, a CSP's ability to leverage the complementarity of the standards does not justify their creators' failure to make a single standard that offers full protection, as CSPs incur significant costs in obtaining multiple certifications. Partial protection could be both acceptable and justifiable for sectoral standards whose goals are specific enough that completeness is unnecessary, but standards that have a general intent of guaranteeing IT security and privacy cannot overlook baseline controls.

The origin of each standard undoubtedly plays a role in its unique characteristics. FedRAMP, designed by the federal government for the federal government, can rely on provisions in federal laws and regulations to complete the baseline. A CSP, as a contractor to the federal government, might have additional obligations that compensate for omissions, as shown by the case of the omitted controls on interoperability and portability of data and applications. An assessment of these obligations prior to the adoption of a standard helps one understand its limits and supports planning for additional controls where necessary. SOC 2 shows its limits in not being a full certification, but rather a form of guidance for assessing IT security. On the one hand, TSPC's greater flexibility has a negative impact on completeness, and a careful strategy is needed in order to define the objectives and specifications of an SOC 2 assessment. On the other hand, flexibility represents uncharted territory for tenants; they cannot rely on the mere availability of an SOC report from their CSP, but need to do a careful reading of the report to identify the extent to which the assessment it describes actually guarantees cloud security. On the other hand, ISO/IEC 27001 benefits from its mixed approach. As a cross-industry IT certification, it is more general by design; furthermore, as it is more specific than TSPC in dictating the controls to implement during a security assessment, it has the smallest number of omitted controls.

The idea that a good balance between general and specific measures is beneficial to the effectiveness of a standard is reinforced by observing the performance of each standard over time (see Figure 8.1). ISO/IEC 27001, which suffered heavily from not including mobile security controls in its older version and from being outdated by the technology advances of the past 11 years, still shows protection comparable to, if not better than, that offered by the others. Not only is the total number of controls it omits lower than that for TSPC 2014 and FedRAMP rev. 3, but the number of most significant controls omitted after the selection (see Figure 8.2) is equal to that of the most recent release of FedRAMP, and second only to ISO/IEC 27001:2013. In a similar vein, TSPC 2009, which was oriented toward the assessment of cross-industry security but was more specific than its later updates, does not suffer from being 7 years older than its most recent successor, outperforming it in the coverage of the most relevant controls.

Although using a better approach in designing IT security standards is not the only element to consider in order to tackle cloud security, it certainly represents a strong start. Flexibility may be reckoned a weakness at first sight, but can also have value if used correctly, and if strong control measures are implemented to fulfill general criteria. Conversely, less flexibility could hold back a CSP in the implementation of necessary protection measures.

8.5.1 Current Challenges

Consideration of the issues described above opens up uncertainty for both the cloud service provider and the tenants. What certification process could be the most beneficial, and what type of compliance should be attained to assure information in the cloud? Looking at a baseline drawn from a single standard may not be enough, but multiple certifications necessarily entail additional costs for the CSP. External audits are expensive not just because of the involvement of third parties, but also because internal resources will be engaged in the auditing process, raising the cost for the CSP beyond the fees for certification.

The benefit of multiple certifications derives from the different approaches used in the standards and the possibility of leveraging flexible criteria and precise controls together. At the same time, the reciprocal coverage of omitted controls among the standards in our study shows that integrations are possible and that each standard could alone assure better coverage than it currently does. A first step must be to improve current standards with the introduction of measures suitable for covering those areas left unprotected. To ensure the success of this process, the study of threats and vulnerabilities must remain a priority, and the results must be translated into existing standards so that they can be effectively improved and updated.

The importance of maintaining updated standards to cope with current issues and vulnerabilities is apparent when we consider the rapid and constant evolution of technology. The Internet of Things (IoT) is one of the most vivid examples of the innovative forms of data processing that keep springing up, generating unexpected threats. The IoT model implies processing of a massive amount of user data, and is often susceptible to flaws caused by lack of basic attention to patches, updates, or the use of passwords [65]. Connected devices require interaction with cloud systems, since limited storage and processing power could limit their performance; at the same time, they have distinctive characteristics that make them more vulnerable to external attacks [66]. In the IoT scenario, single devices could be the weak link in the chain protecting C-I-A principles, and their violation might have disruptive consequences. Standards must have the resilience to handle the appearance of radical innovations, such as in the interaction of IoT devices and cloud systems. At the same time, the resilience of a standard cannot be determined solely by the frequency of its updates, but rather must be implicit in its design.

An alternative path could be to promote the optimization of the certification processes, and the achievement of better integration of assessment procedures through dialogue with and cooperation of the stakeholders. Choosing this path, and avoiding repetition of controls across standards once a first certification has been obtained, is a necessary step. Initiatives in that direction have been promoted by industry advocacy groups [67] and government agencies [68], but their extent is still unclear and limited to guidance for auditors and CSPs, rather than a full, mutual recognition.

8.5.2 Opportunities

Standards can truly help improve security and privacy. By establishing a security baseline and setting the scene for the implementation of privacy- and security-enhancing controls, standards can optimize the outcomes of CSPs' efforts to ensure information assurance. From that perspective, standards should be seen as a security tool.

In particular, the use of standards can help improve CSPs' efficiency in implementing security and privacy measures. A CSP trying to protect its cloud system without guidance could overprotect certain areas while overlooking the need for other measures. By improving the security controls in relation to the threat landscape, standards can help a CSP achieve the best possible security result while lowering assessment and implementation costs.

Our observations on the impact of the “Treacherous Twelve” on the effectiveness of IT security standards in cloud environments point toward a possible path to the improvement of IT security standards. Observations of threats, issues, and vulnerabilities can help us understand the need for new or different control measures, and their connection to security standards can lead to better effectiveness, completeness, and efficiency.



  1. 1 Mell, P. and Grance, T. (2011) The NIST definition of cloud computing: recommendations of the National Institute of Standards and Technology, Special Publication 800-145, National Institute of Standards and Technology, U.S. Department of Commerce, September. Available at
  2. 2 Fischer, E.A. and Figliola, P.M. (2013) Overview and issues for implementation of the Federal Cloud Computing Initiative: implications for federal information technology reform management. Journal of Current Issues in Media and Telecommunications, 5 (1), 1–27.
  3. 3 Harms, R. and Yamartino, M. (2010) Economics of the cloud, Microsoft, Nov. 11. Available at
  4. 4 Office of Management and Budget (2017) Analytical Perspectives. Budget of the U.S. Government. Available at Retrieved from
  5. 5 Kundra, V. (2011) Federal cloud computing strategy. Available at (accessed May 27, 2016).
  6. 6 Gartner (2016) Gartner says worldwide public cloud services market to grow 17 percent in 2016, Sep. 15. Available at (accessed Nov. 20, 2016).
  7. 7 Inci, M.S., Gulmezoglu, B., Irazoqui, G., Eisenbarth, T., and Sunar, B. (2015) Seriously, get off my cloud! Cross-VM RSA key recovery in a public cloud, Cryptology ePrint Archive, Report 2015/898, Sep. 22. Available at (accessed Oct. 9, 2016).
  8. 8 Liu, F., Yarom, Y., Ge, Q., Heiser, G., and Lee, R.B. (2015) Last-level cache side-channel attacks are practical, in Proceedings of the IEEE Symposium Security and Privacy, pp. 605–622.
  9. 9 Zhang, Y., Juels, A., Reiter, M.K., and Ristenpart, T. (2012) Cross-VM side channels and their use to extract private keys, in Proceedings of the ACM Conference on Computer and Communications Security, pp. 305–316.
  10. 10 Zhang, Y., Juels, A., Reiter, M.K., and Ristenpart, T. Cross-tenant side-channel attacks in PaaS clouds, in (2014) Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, pp. 990–1003.
  11. 11 Verizon (2016) 2016 data breach investigations report. Available at
  12. 12 Timmermans, S. and Epstein, S. (2010) A world of standards but not a standard world: toward a sociology of standards and standardization. Annual Review of Sociology, 36, 69–89.
  13. 13 Brunsson, N. and Jacobsson, B. (2000) The contemporary expansion of standardization, in A World of Standards (eds. N. Brunsson and B. Jacobsson), Oxford University Press, pp. 1–18.
  14. 14 International Organization for Standardization (ISO) About ISO, ISO. Available at (accessed Jun. 1, 2016).
  15. 15 National Institute of Standards and Technology (NIST) (2016) About NIST, Aug. 25. Available at (accessed Nov. 30, 2016).
  16. 16 Circular No. A-119, Revised: Federal Participation in the Development and Use of Voluntary Consensus Standards and in Conformity Assessment Activities, Circular No. A-119, revised, National Institute of Standards and Technology (NIST), U.S. Department of Commerce, Jan. 27. (2016) Available at (accessed May 17, 2017).
  17. 17 Schroeder, F.E.H. (1986) More “small things forgotten”: domestic electrical plugs and receptacles, 1881–1931. Technology and Culture, 27 (3), 525–543.
  18. 18 Drake J., (2015) Stagefright: scary code in the heart of Android: researching Android multimedia framework security, Black Hat, USA, Aug. 5. Available at (accessed Oct. 15, 2016).
  19. 19 Goodin, D. (2013) Google confirms critical Android crypto flaw used in $5,700 Bitcoin heist, Ars Technica, Aug. 14. Available at (accessed Jun. 8, 2016).
  20. 20 Ormandy, T. (2007) An empirical study into the security exposure to host of hostile virtualized environments. Available at
  21. 21 Defense Information Systems Agency (2017) Department of Defense cloud computing security requirements guide. Available at (accessed May 17, 2017).
  22. 22 DigitalGov (2016) FedRAMP high baseline requirements for federal agencies, YouTube, Jun. 30. Available at (accessed Nov. 20, 2016).
  23. 23 U.S. Department of State (2016) 12 FAM 540: sensitive but unclassified information (SBU), CT:DS-256, Foreign Affairs Manual, Apr. 7. Available at (accessed Nov. 20, 2016).
  24. 24 Kundra, V. (2010) 25 Point implementation plan to reform federal information technology management, The White House, Washington, DC, Dec. 9. Available at (accessed May 23, 2016).
  25. 25 American Institute of Certified Public Accountants (AICPA) (2014) Service Organization Control reports. Available at (accessed May 16, 2017).
  26. 26 McClure, D. (2010) Statement of Dr. David McClure, Associate Administrator, Office of Citizen Services and Innovative Technologies, General Services Administration, before the House Committee on Oversight and Government Reform Subcommittee on Government Management, Organization, and Procurement, U.S. General Services Administration (GSA), Jul. 1. Available at (accessed Nov. 24, 2016).
  27. 27 VanRoekel, S. (2011) Security authorization of information systems in cloud computing environments. Available at (accessed May 17, 2017).
  28. 28 FedRAMP (2014) Guide to understanding FedRAMP, version 2.0, Jun. 6. Available at (accessed June 6, 2014).
  29. 29 European Telecommunications Standards Institute (ETSI) (2013) Cloud standards coordination: final report, version 1.0, November. Available at (accessed May 25, 2016).
  30. 30 Joint Task Force Transformation Initiative (2010) Guide for Applying the Risk Management Framework to Federal Information Systems: A Security Life Cycle Approach, NIST Special Publication 800-37, Revision 1, National Institute of Standards and Technology (NIST), U.S. Department of Commerce, February. (with updates as of Jun. 5, 2014). Available at (accessed Aug. 2, 2017).
  31. 31 International Organization for Standardization (ISO) (2012) ISO/IEC 17020:2012: Conformity Assessment – Requirements for the Operation of Various Types of Bodies Performing Inspection, March. Available at (accessed Nov. 24, 2016).
  32. 32 FedRAMP (2016) Program overview. Available at (accessed May 26, 2016).
  33. 33 Joint Task Force Transformation Initiative (2013) Security and privacy controls for federal information systems and organizations, NIST Special Publication 800-53, Revision 4, National Institute of Standards and Technology (NIST), U.S. Department of Commerce, April. Available at (accessed May 31, 2016).
  34. 34 Computer Security Division, Information Technology Laboratory, National Institute of Standards and Technology (NIST) (2004) Standards for security categorization of federal information and information systems, Federal Information Processing Standards Publication FIPS PUB 199, February. Available at (accessed May 31, 2016).
  35. 35 FedRAMP High Baseline Release, Jun. 17. (2016) Available at (accessed Nov. 24, 2016).
  36. 36 FedRAMP (2016) Marketplace. Available at (accessed Nov. 21, 2016).
  37. 37 American Institute of Certified Public Accountants (AICPA) (2016) About the AICPA. Available at (accessed Nov. 26, 2016).
  38. 38 Gartner (2010) Gartner says SAS 70 is not proof of security, continuity or privacy compliance, Jul. 14. Available at (accessed Nov. 25, 2016).
  39. 39 Nickell, C.G. and Denyer, C. (2007) An introduction to SAS 70 audits. Benefits Law Journal, 20 (1), 58–68.
  40. 40 American Institute of Certified Public Accountants (AICPA) (2011) New SOC reports for service organizations replace SAS 70 reports, Feb. 7. Available at (accessed Nov. 24, 2016).
  41. 41 International Organization for Standardization (ISO) ISO/IEC JTC 1 – Information Technology. Available at (accessed Jun. 1, 2016).
  42. 42 International Organization for Standardization (ISO) ISO/IEC 27000 family: information security management systems, ISO Available at (accessed Jun. 1, 2016).
  43. 43 Charlet, L. (2015) ISO Survey, International Organization for Standardization (ISO). Available at (accessed Nov. 24, 2016).
  44. 44 Gantz, S.D. (2014) The Basics of IT Audit: Purposes, Processes, and Practical Information, Syngress, Waltham, MA.
  45. 45 Bird, K. (2013) New version of ISO/IEC 27001 to better tackle IT security risks, International Organization for Standardization (ISO), Aug. 14. Available at (accessed May 29, 2016).
  46. 46 Watkins, S.G. (2013) An Introduction to Information Security and ISO27001:2013: A Pocket Guide, 2nd edn, IT Governance Publishing, Ely, UK.
  47. 47 PCI Security Standards Council, LLC PCI Security. Available at (accessed Nov. 30, 2016).
  48. 48 Montanari, M., Huh, J.H., Dagit, D., Bobba, R.B., and Campbell, R.H. (2012) Evidence of log integrity in policy-based security monitoring, in 2012 IEEE/IFIP 42nd International Conference on Dependable Systems and Networks Workshops, Article No. 6264693.
  49. 49 Wright, S. (2011) PCI DSS: A Practical Guide to Implementing and Maintaining Compliance, 3rd edn, IT Governance Publishing, Ely, UK.
  50. 50 Bundesamt für Sicherheit in der Informationstechnik (BSI) (2016) Anforderungskatalog Cloud Computing (C5). Available at (accessed Nov. 29, 2016).
  51. 51 Cloud Security Alliance (CSA) (2016) About. Available at (accessed May 24, 2016).
  52. 52 Cloud Security Alliance (CSA) CSA STAR: the future of cloud trust and assurance. Available at (accessed May 31, 2016).
  53. 53 Cloud Security Alliance (CSA) Introduction to the Cloud Control Matrix Working Group. Available at (accessed May 21, 2016).
  54. 54 Cloud Security Alliance (CSA) (2016) ‘The Treacherous Twelve’ cloud computing top threats in 2016. Available at (accessed May 23, 2016).
  55. 55 Bayuk, J.L. (2011) Alternative security metrics, in Proceedings of the 8th International Conference on Information Technology: New Generations, pp. 943–946.
  56. 56 Adobe (2015) Adobe security and privacy certifications, Adobe Systems Incorporated. Available at (accessed Jun. 1, 2016).
  57. 57 Microsoft Simplify compliance with the Microsoft Common Controls Hub, Microsoft Trust Center. Available at (accessed Nov. 30, 2016).
  58. 58 Ardagna, C.A., Asal, R., Damiani, E., and Vu, Q.H. (2015) From security to assurance in the cloud: a survey. ACM Computing Surveys, 48 (1), Article 2.
  59. 59 Cloud Security Alliance (CSA) (2010) Top threats to cloud computing V1.0. Available at (accessed May 23, 2016).
  60. 60 Cloud Security Alliance (CSA) (2013) The Notorious Nine: Cloud Computing Top Threats in 2013. Available at (accessed May 24, 2016).
  61. 61 Di Giulio, C., Kamhoua, C., Campbell, R.H., Sprabery, R., Kwiat, K., and Bashir, M.N. (2017) Cloud standards in comparison: are new security frameworks improving cloud security? in Proceedings of the IEEE 10th International Conference on Cloud Computing (CLOUD), pp. 50–57.
  62. 62 Di Giulio, C., Kamhoua, C., Campbell, R.H., Sprabery, R., Kwiat, K., and Bashir, M.N. (2017) IT security and privacy standards in comparison: improving FedRAMP authorization for cloud service providers, in Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp. 1090–1099.
  63. 63 Di Giulio, C., Sprabery, R., Kamhoua, C.A., Kwiat, K., Campbell, R.H., and Bashir, M.N. (2017) Cloud security certifications: a comparison to improve cloud service provider security, in Proceedings of the 2nd International Conference on Internet of Things, Data and Cloud Computing (ICC), ACM, New York, NY.
  64. 64 Cloud Security Alliance (CSA) FedRAMP cloud controls matrix v3.0.1 candidate mapping. Available at (accessed Jun. 1, 2016).
  65. 65 Roos, G. (2015) How cloud, IoT are altering the security landscape, Channel Insider, Sep. 13. Available at (accessed May 17), 2017.
  66. 66 Fraga-Lamas, P., Fernández-Caramés, T.M., Suárez-Albela, M., Castedo, L., and González-López, M. (2016) A review on Internet of Things for defense and public safety. Sensors, 16 (10), Article 1644.
  67. 67 FedRAMP Fast Forward Industry Advocacy Group Fix FedRAMP: a 6-point plan. Available at
  68. 68 U.S. Government Accountability Office (GAO) (2010) Report to the Chairwoman, Subcommittee on Government Management, Organization, and Procurement, Committee on Oversight and Government Reform, House of Representatives: Information Security: Progress Made on Harmonizing Policies and Guidance for National Security and Non-National Security Systems, GAO-10-916, Washington, DC, September. Available at