Gary Smith, VP of Portfolio Security & Art Corcoran, Director of Security Operations
December 5, 2019

Data centers are entering a new transformational era. Gone are the days of air-gapped cyber and physical security. Data centers are monitored and managed through a network of hundreds or even thousands of sensors used for real-time telemetry—heating and cooling, maintenance alerts, physical security, and much more. Yet, in addition to introducing new opportunities for enhanced operational efficiencies and greater visibility and control, digital transformation (DX) presents new challenges. Manipulation of heating and ventilation controls (HVAC) could result in critical infrastructure systems being shut down or compromised. Physical cameras could be hacked and commandeered to disguise a robbery or unauthorized entry into a secure location.

Data centers must be diligent to ensure physical and cyber systems are protected in a unified way, and that their convergence does not create additional risks. Further, data centers will begin to see advanced technologies such as artificial intelligence (AI) and machine learning (ML) deployed to pinpoint anomalies in both physical and cyber security and to enact real-time controls and remediation processes.

Digital transformation presents new data center challenges

The data center for many businesses is seen as a strategic lever in their efforts to support business acceleration requirements. Business acceleration objectives include:

Addressing emerging markets
Lowering costs while improving operating efficiencies
Creating more and better customer engagement
Tapping new revenue opportunities

Remaining competitive and maintaining aggressive margins requires ongoing innovation, and DX stands at the center. The data center plays a pivotal role in enabling many DX initiatives. Data centers are where the cloud lives. Leveraging this fact allows data centers to become private cloud service centers that streamline processes while retaining control of operations and security. These give organizations the scale required to tackle these new DX initiatives—the compute power and storage capabilities to power the new applications and accompanying data that is generated. A significant factor driving this is the growing reliance of AI and ML on big data.

To address this new data center landscape, many organizations are opting to use colocation facilities rather than building out their own data centers. The colocation data center market is growing at a 15.4% compound annual growth rate (CAGR) from 2016 through next year. But just as the new evolution of the data center is helping to enable DX, the data center is also undergoing a DX transformation—one that has its own challenges.

Nearly 80 percent of organizations are introducing digital transformation faster than their ability to secure them against cyber attacks.

Colocation creates a convergence of physical and cyber security

It’s critical that physical and cyber security be approached in a unified manner since they have become increasingly convergent in recent years. The most obvious example of this is the fact that physical security deployments (card readers, CCTV) now all reside on the network. This results in a physical security dependency on strong cybersecurity controls to ensure the integrity of physical security infrastructure. However, this dependency in turn offers intriguing new ways to leverage both disciplines to drive new security paradigms. For example, if an employee logs into a computer in San Francisco yet physically accesses a New York data center, that would constitute a major red flag. With artificial intelligence and machine learning, the system could adapt to such a situation to alert responders and revoke access permissions immediately without human intervention. But with technical innovation comes new attack vectors and vulnerabilities, for the truth is that hackers are always looking for undetected paths into your systems. Many traditional physical devices now represent threats to cyber security. Things like ID cards, biometrics, HVAC, laptops, smartphones, and USBs are all cyber security threats since they are now IP-enabled. In fact, IoT devices experience an average of 5,200 attacks per month.

DX requires a rethink of physical security. However, the need to focus on controlled access, meaning designated workers should be able to access only the areas, systems, and applications to which they should have access, remains the foundational concept of ensuring the confidentiality, integrity and availability of data wherever it resides and regardless of how it is used.

To ensure adherence to these access protocols, organizations need to institute digital and physical monitoring—which must occur at the rack level and provide a complete compliance audit trail, full transparency and reporting, and automated processes for revoking access. Regulations that may apply include the Payment Card Industry Data Security Standard (PCI DSS), Health Insurance Portability and Accountability Act (HIPAA), the Federal Information Security Management Act (FISMA), the European Union’s General Data Protection Regulation (GDPR), and Sarbanes-Oxley Act 2002 (SOX). But organizations should not stop at government and industry regulations; they should also consider implementing security standards such as the National Institute of Standards and Technology (NIST), the International Organization for Standardization (ISO), the Center for Internet Security (CIS) controls, and others.

AI and ML provide the means to monitor physical security devices such as doors and cameras, pinpointing anomalies, sending real-time alerts to data center personnel, and even acting as a digital system to immediately combat threats without human intervention. As noted at the beginning of this section, the convergence of cyber and physical security allows physical security alerts to activate cybersecurity protocols such as blocking access to data and systems for users, devices, and applications based on predetermined business rules. As this technology matures and becomes ubiquitous, it will shift the security paradigm from one of “detect and respond” to one of “prevent and counter”.

A physical security data center checklist

When vetting colocation facilities, organizations need to evaluate solutions based on criteria that include questions such as:

Is the data center redundant? It is important for your data center to be prepared for the unexpected. Any number of things could go wrong such as utility failure, cooling system equipment failure, fire, air quality issues and natural disasters. That is why building out a redundant design for your data center is crucial. There are three redundancy maturity models. The first involves building redundancy across all system elements which are critical and must deliver the capacity required to power, backup, and cool a facility at full IT load. But achieving a full IT load is not enough in the event of a component failure or if a system element must undergo maintenance. In order to address these events that cannot be predicted, data center design calls for at least one independent backup unit for every 4 needed (the second maturity model is referred to as N+1). The final maturity model, 2N, refers to redundancy where there are two independent distribution systems—from power supplies to cabling. Finally, does the redundancy model extend beyond power, cooling and backup to include the physical security infrastructure? If not, a significant exposure to the security of your data can be as close as the next power outage.
Is the building constructed to withstand external attacks or natural disasters? Wall density should be designed to withstand explosive devices and natural elements. Windows should be minimized, used only for public spaces and employ shatter resistant window film. A buffer of at least 100 feet should extend around the site to protect from vehicles and, ideally, security guards/stations should be employed for access. Kevlar fire-resistant walls are also an important requirement.
Are entry and exit points limited? There should be one main entrance to the building and a loading dock, typically located at the rear of the building. Vehicle pathways leading to entrance areas should be blocked by crash-resistant bollards, industrial concrete planters or other barriers to prevent vehicle penetration at these critical access points. Fire doors should be exit-only and both entry points should be monitored 24x7 using IP-enabled video surveillance. These cameras should be integrated into the network firewall to ensure they are protected from cyber-attacks.
What physical intrusion detection policies are in place? Automated electronic intrusion detection systems, including event-driven closed circuit television cameras (CCTV) and alarms, are a requisite. Here, data center teams must have documented policies for response to ingress and egress violations.
What security surveillance cameras are in use? Camera systems should be tailored to their application. This may include motion-detection, pan-tilt-zoom, and low-lighting capabilities. They also need to be integrated into network security with passwords and credentials designated for access (viz., IP-enabled) and isolated by data center firewalls to ensure they cannot be compromised, or be used to compromise the internal data center network. Organizations also need to implement data retention and destruction policies for surveillance footage. These must comply with relevant laws, industry regulations and IT standards. We would suggest retaining surveillance video footage for a minimum of 90 days.
Is multi-factor authentication employed? Ingress and egress access must be controlled by multi-factor authentication. Biometric identification helps ensure that personnel access only those areas to which they are authorized. Given privacy concerns about the sensitivity of biometric data, it is recommended that biometric data remain in the possession of the end-user, e.g., the biometric algorithm should reside directly on the user’s credential as opposed to a central database.

Are hardened access layers used? Any person who enters the most secure area of a data center should be required to authenticate at least four times—for example, building perimeter entrance to lobby/loading dock, lobby/loading dock entrance to common space, entrance from common space to data center space and entrance to the most secure area (cage, cabinet, etc.).

Cyber threats to the data center

The biggest challenge to data center security today is not physical threats but rather cyber threats. The proliferation of applications and burgeoning mounds of intellectual property and private information—often governed by regulators—makes data centers a central target for cyber criminals and even nation-states.

Defending against data center attacks

As a result of DX, the cyber-attack surface for the data center is expanding and becoming increasingly harder to defend. These threats can target physical devices and systems used to manage cooling and video surveillance, among others. They can also target personnel through spear phishing, gaps in authentication protocols, and other malicious means.

Unless data-center vulnerability is internet-facing, attackers must be persistent and employ advanced strategies to achieve a successful exploitation:

Implement two-factor or multi-factor authentication. Many data centers rely on local authentication options in the event of an emergency. These local authentication channels are not logged and the same login credentials are often shared across hosts and workloads (for simplicity). This exposes them to bad actors, who, once they have stolen them, can use them to gain access to the data center. Adding multiple layers of authentication for a single user through two-factor or multi-factor authentication will ensure a higher level of security, making it much more difficult for an intruder to access systems they are not allowed to access.
Target known vulnerabilities with patching and updates. Virtualized environments and resources must still run on physical hardware—specifically, virtual disks are dependent on physical disks that reside on physical servers. Management planes have their own management protocols, power, processors, and memory that are managed via protocols such as Intelligent Platform Management Interface (IPMI). These latter protocols reside beneath virtualization layers and are slow to receive updates and patches. Known to have security weaknesses, bad actors target vulnerabilities in IPMI. Organizations must ensure they practice good cyber hygiene and that their patching and updates target known vulnerabilities being targeted by cyber criminals.
Build barricades. Threats from outside the data center such as email, web gateways, DevOps, Internet of Things (IoT), and operational technology (OT) present substantial risk, and cyber criminals are exploiting each of these attack surfaces. Here, lateral (east-west) movement of malicious intrusions allow cyber criminals to gain access to the data center. All of these present serious risk to the data center; consider some of the latest trends:
94 percent of data malware actions are tied back to email
77 percent of critical infrastructure (viz., OT) organizations reported at least one security breach in the past year
25 percent of cyber attacks in 2020 will target IoT devices

An emerging trend is the move to augment point-in-time penetration tests designed to validate cyber integrity with a reliable continuous monitoring capability that operates in real time. Tools are available to analyze incoming network traffic for anomalies and identify those which require more scrutiny by information security engineers. This builds assurance that network security is operating effectively over prolonged periods of time and serves to validate pen test results.

Build a virtual or a digital system. The reality is that no matter how much you try to protect against security breaches from coming in, it’s becoming increasingly more and more difficult to do so. Threats can come in the form of hacked devices, such as servers, routers, switches, and firewalls. In these instances, known vulnerabilities are targeted in these devices, employing rootkits that sit below the operating system and are hard to detect. Ironically, the very devices intended to protect an enterprise are infected and turned into malicious gateways into the data center. That’s why AI and ML are starting to be implemented as security strategies to act more like an immune system by detecting and fighting threats from within instead of purely focusing on keeping threats out at the perimeter. These strategies can act like antibodies in the human body to combat suspicious behavior that falls outside the norm without shutting the entire system down.

Mapping out the most common data center attack vectors

Data centers are attractive, lucrative targets for cyber criminals and nation-states. There are a number of reasons:

Financial gain and notoriety
Theft of intellectual property (IP)
Theft of private customer data—often used to instigate subsequent attacks (resulting in identity theft, stolen money from bank accounts, credit card fraud, etc.)
Financial losses due to operational outages and brand damage
Social and environmental damage (particularly true in the case of nation-state attacks on OT)
In the case of nation-states, compromise the national security assets of a geopolitical rival

The average cost of a data center outage has increased by almost 38 percent—hitting $9,000 per minute.

These different attack objectives are achieved through the use of various attack vectors. The following are some of the most prevalent:

Distributed denial of service (DDoS) attacks. In addition to being on the receiving end of DDoS attacks whereby criminals seek to disrupt and disable essential internet services, web servers are being turned into bots to attack other websites and gain access into data center environments. DDoS is a serious problem, with attacks increasing in volume and sophistication in 2018. Research by IDC, as an example, reveals that half of IT security leaders indicate their organizations were a victim of DDoS attacks as many as 10 times in the past year. And these were sustained attacks, with 40 percent lasting over 10 hours.
Web application attacks that leverage vulnerabilities such as SQL injection, cross-site scripting, and cross-site request forgery are employed to break into applications and steal data for profit. With DevOps activities ramping up across most businesses, and many hosted in private cloud environments, the cybersecurity protections for applications must extend from pre-deployment to post-deployment. Research shows DevOps organizations have significant holes: 46 percent are confronting security risks up front, and only half are fixing major vulnerabilities.
DNS infrastructure: attack target and collateral damage. DNS servers can be taken offline very easily, thus keeping thousands of users from accessing the internet. DNS servers can also be used to amplify the impact of a DDoS attack (viz., drown victims with DNS traffic).
Secure sockets layer (SSL)/transport layer security (TLS)-induced security blind spots. Bad actors are turning to SSL/TLS to infiltrate data centers by wrapping and protecting the delivery of malicious payloads and then shielding data exfiltration upon successful intrusion. Many firewalls are not designed to scale with the additional requirements that SSL inspection requires, and organizations need to evaluate firewall performance capabilities.
Brute force and weak authentication. Applications employ authentication to verify user identity. Single-factor authentication (such as username and password only) is weak and criminals can use brute force to hack credentials. As covered above, data centers must employ multi-factor authentication to ensure bad actors are unable to bypass authentication controls.
Integrate operational technology (OT) and informational technology (IT). Traditionally, OT and IT have been siloed as separate organizational functions. Operations kept the data centers running while IT managed business applications from the office. However, with technology rapidly changing and the convergence of physical and cyber, it’s critical that OT and IT teams work hand-in-hand to prevent and combat security threats.

Measuring the impact of operational disruptions and data breaches

Data center disruptions can have a dramatic impact on operations, quickly tallying into the hundreds of thousands of dollars. And this does not include brand repercussions. The severity of DDoS attacks is likely to continue to rise. Not only are they growing in frequency, but they are increasing in size (with new records set several times in 2018). Further, with fewer and fewer OT systems air-gapped, the attack surface for operational disruption and disablement becomes larger while the operational repercussions become dramatically greater.

The cost of these operational disruptions can spiral very quickly. In 2016, the average cost per minute per data center outage was $8,851 and most outages usually last longer than a minute. In fact, the average total cost of an outage is $740,000 which would translate to an average of one hour and twenty minutes of downtime.

Data breaches are just as problematic for data centers. Data breaches equate to an average of $150 per record. On average, a data breach costs a company over $8 million in the U.S. which has increased slightly from 2018 and is more than twice the global average.

Attack kill chain: Adversaries vs. Defenders

The attack kill chain is used by security professionals to understand the flow of an attack and the cybersecurity strategies needed to defend each stage. The following attack kill chain flow derives from a model developed by Lockheed Martin.

Kill Chain Stage	Adversary (Bad Actor)	Defenders
1. Reconnaissance Objective: Identify the Targets	Harvest information to understand which targets will enable them to meet their objectives (email addresses, internet-facing servers, etc.).	Detect reconnaissance to determine intent of adversary (e.g., collect and analyze visitor logs, detections based on browser analytics, etc.)
2. Weaponization Objective: Prepare the Operation	Create malware deliverable payload; often use malware-as-a-service and other dark net tools (includes backdoor implant for command and control, mission ID, etc.).	Analyze malware artifacts to develop signatures for detection.
3. Delivery Objective: Launch the Operation	Launch attack that targets web servers or comes through a specific delivery channel such as email, USB stick, social media, water holes (viz., compromised websites).	Block intrusion attempts by collecting email and web logs for forensic reconstruction, understanding delivery medium and targeted servers and people, etc.
4. Exploitation Objective: Gain Access to the Victim	Exploit a software, hardware, or human vulnerability with known or unknown (zero-day) vulnerability (server-based or victim triggered).	Harden systems for resiliency and use behavior and machine learning (ML)-based detection to stop unknown vulnerabilities. Includes threat intelligence to prioritize vulnerability patching and endpoint auditing and management.
5. Installation Objective: Establish a Beachhead at the Victim	Install persistent backdoor or an implant to maintain access (“time stomp” to make malware appear as part of the standard operating system, etc.).	Implement endpoint instrumentation to detect and log malicious activity as well as block designated activity (discover abnormal file creations, extract certificates of any signed executables, understand time of malware intrusion, etc.).
6. Command & Control Objective: Remotely Control the Implants	Malware opens a command channel that enables adversary to manipulate the victim (e.g., two-way communications channel over web, DNS, and email protocols).	Blocks Command & Control through malware analysis, proxy category blocks, DNS sink holing and name server poisoning, etc.
7. Actions on Objectives Objective: Achieve the Mission’s Goal	Successful infiltration results in any number of actions: a) collect user credentials, b) privilege escalation, c) internal reconnaissance, d) lateral (east-west) movement through an IT environment, e) collect and exfiltrate data, f) destroy systems, g) overwrite or corrupt data, or h) surreptitiously modify data.	The longer an adversary has access, the greater the damage that can be enacted. Uses forensic evidence such as network packet captures to determine damage impact and enact incident response play

Successful infiltration results in any number of actions: a) collect user credentials, b) privilege escalation, c) internal reconnaissance, d) lateral (east-west) movement through an IT environment, e) collect and exfiltrate data, f) destroy systems, g) overwrite or corrupt data, or h) surreptitiously modify data.

The longer an adversary has access, the greater the damage that can be enacted. Uses forensic evidence such as network packet captures to determine damage impact and enact incident response playbook.

Per NSS Labs, HTML injection is the most frequently reported data center attack.

Future of data center security

Keeping pace with the rapidly evolving threat landscape necessitates a security program that is comprehensive, integrated, and employs advanced technologies. This approach encompasses cybersecurity and physical security as both are important. With data centers—a critical lever for DX initiatives and private cloud adoption growing their footprints in many instances—successful exploitation of these data center threats can have serious ramifications.

Looking to the future, data center leaders need to embrace additional cyber and physical security strategies. At the forefront and reaching across the entire security fabric is the integration of cyber and physical security. Attacks are becoming increasingly multi-stage, targeting physical security through cyberattacks that create physical exposure. And with 34% of attacks involving internal players, physical security remains critical. The moral here is that cyber and physical security are complementary parts of a complete security program.

Data centers need to ensure that their cyber and physical security is seamlessly integrated. Physical systems and devices must reside on secure networks and behind firewalls. This helps protect them from malicious attacks, while providing seamless incident response capabilities in the event of an intrusion.

Other security strategies that data center leaders should have in place include:

Data governance—at rest and in transit, across and between multiple cloud environments. Most enterprises are experiencing 40-50% annual growth in unstructured data. To protect this information, whether on-premises or the cloud, organizations need to implement data governance policies in control—for moving data across and between different environments and between applications.
Cloud transparency and controls. For public clouds, organizations need to ensure they have the right governance policies and controls in place. These are important. Gartner predicts that 60 percent of enterprises with cloud governance will experience 33 percent fewer security incidents.
Security integration. 83 percent of IT leaders cite organizational complexities as putting them most at risk. Only 48% have security policies in place to manage data access amongst employees and third parties. A new, integrated security framework is needed. Traditional security architectures are fragmented, and it is difficult to share information across and between the different elements. This includes new data center attack surface areas such as DevOps and the cloud for full transparency and centralized controls.
Protecting the edge of network. 5G increases the ease and speed at which devices attach to the network as well as the amount of data that can be accessed and moved. IoT poses substantial risk (as these devices cannot be managed via traditional security models), and lateral intrusions can impact data center security. Software-defined wide area networks (SD-WAN) leverage 5G as an additional bandwidth channel, which bypasses traditional data center security controls. This increases risk that can back-funnel into the data center via lateral movement.
Threat intelligence: artificial intelligence and machine learning. 85% of organizations indicate threat intelligence is critical to a strong security posture. But only 42% believe they are very effective in using threat intelligence. Part of the problem is lack of in-house expertise (50%). To keep pace with security threats that are using artificial intelligence (AI) and machine learning (ML) and are polymorphic and multi-vector, cybersecurity leaders must employ ML and AI capabilities themselves, or switch to tools that have AI/ML enhanced capabilities. This enables them to reduce the attack surface for prevention, detection, and remediation.

State of data center security per NSS labs

70 percent of organizations indicate data-center security capabilities are cloud delivered
50 percent of organizations still deploy physical security appliances on-premises
80 percent use IPS to block DDoS attacks
90 percent use anti-malware, web application firewalls (WAFs), and stateful firewalls

Turning the data center into a DX enabler

DX is propelling business acceleration, and the data center is the engine making much of it possible. But with this expanded attack surface, also comes greater threats to the data center—both physical and cyber. DX is also driving a transformation of the data center that presents new security challenges.

To protect their environments from these new and expanded threats, IT leaders must ensure they have the right defenses in place. The convergence of physical and cyber threats necessitates the integration of data center security. Here, IT leaders need to ensure their physical systems and devices are integrated into network security and behind firewalls. Finally, to counter advances in the threat landscape, data centers need to tap cybersecurity that leverages AI and ML capabilities.

CBRE Access More Productive Space: Convert Power Demanding Computer Space to Productive Real-Estate