.png)
In a recent discussion, we explored the intersection of NERC CIP regulations and cloud technology with Tom Alrich. Tom is a longtime NERC CIP expert and writes a highly regarded blog covering NERC CIP in the cloud and software supply chain security, including software bills of materials (SBOM). Our conversation covered the explicit and implicit constraints on cloud usage within the energy sector, the difference between "systems" and "individual assets," along with the challenges around providing compliance evidence for the cloud. Watch the interview and/or read below to understand the latest on cloud and NERC CIP.
What exactly is the problem (or problems) with NERC CIP and the cloud, anyway?
The problem is that medium and high impact BES Cyber Systems (BCS), Electronic Access Control or Monitoring Systems (EACMS), and Physical Access Control Systems (PACS) effectively are not allowed to be implemented in the cloud today. Also, if a NERC entity, which owns or operates a medium or high impact CIP environment, utilizes a service in the cloud that functions as one of these types of systems, they will also be in violation – just as they would if they had implemented the system in the cloud themselves.
Why is this the case? Do the CIP requirements prohibit using cloud-based medium or high impact systems?
Most people assume that’s the case. In other words, they assume that the wording of the standards somehow prohibits use of the cloud for these systems. However, the current CIP standards say nothing at all about the cloud. This is because the current CIP framework (with BES Cyber Systems classified as high, medium and low impact, etc.) was first formulated in 2009. At that time, use of the cloud was still considered highly experimental by FERC, NERC, and probably most NERC entities. At that time, nobody even considered the idea that there would ever be a time when even FERC would agree that safe use of the cloud was possible and even desirable.
Since the CIP standards don’t mention the cloud at all, a NERC entity that wants to implement a medium or high impact BES Cyber System in the cloud is free to do so. However, in doing that, they will still be violating maybe 10-20 CIP requirements. The reason this will happen is that the cloud service provider (CSP) will never be able to provide the NERC entity with the documentary evidence that’s required to prove compliance. As anyone involved with NERC compliance (and not just with CIP compliance) knows, “If you didn’t document it, you didn’t do it.”
Why can’t the CSP provide compliance evidence?
This gets back to a fundamental problem with CIP that’s seldom acknowledged: There are a lot of implicit requirements that don’t appear anywhere in the CIP standards, yet if a NERC entity doesn’t perform one of those implicit requirements, they’ll be in violation of one of the explicit requirements – i.e., the ones that are written down.
Here’s a simple example: There are lots of CIP requirements that apply to EACMS (electronic access control or monitoring systems), PCAs (protected cyber assets), and PACS (physical access control systems), yet there is no requirement to identify such systems in the first place. In theory, a NERC entity could tell their auditor that their audit scope for say CIP-007 R2 (patch management) doesn’t apply to EACMS because they’ve never identified any EACMS. And that was because there’s no requirement to do so. No NERC entity would try something like that, since they know they’d get slammed by the auditors – even if they were technically correct.
Another implicit requirement has to do with the fact that BES Cyber Systems are composed of BES Cyber Assets (or BCAs). BES Cyber System is a nebulous concept that just means “a combination of BES Cyber Assets”, whereas BCAs are individual devices like servers or relays. However, there isn’t a single CIP requirement that mentions BCAs at all.
For a lot of requirements that apply to BCS, like the requirements for cyber security policies or cyber security training, it’s fine that BES Cyber Assets aren’t mentioned, since they play no role in policies or training. However, for requirements like CIP-007 R2 patch management and CIP-010 R1 configuration management, individual devices must be taken into account – there’s no other way to comply with the requirement.
For example, you don’t patch a “system”, meaning a combination of cyber assets. You patch each cyber asset individually (even if you’ve automated the task). If you have a BES Cyber System that consists of 100 BES Cyber Assets and you don’t follow the CIP-007 R2 procedures for just one of those BCAs for one month, you’ll be in violation for the whole BCS, just as if you hadn’t patched the other 99 BCAs either.
This shows that, even though the requirement may apply to BCS, not BCAs, in some cases – like in CIP-007 R2 and CIP-010 R1 - there’s an implicit requirement that says each individual BCA is in scope.
Now let’s go back to the question of evidence for BES Cyber Systems in the cloud. Let’s assume a NERC entity has implemented a single BCS in the cloud. For many of the CIP requirements like CIP-003 R1, the requirement for cyber security policies, the fact that the BCS is made up of individual devices doesn’t matter, since that fact is irrelevant to compliance. The system could just as easily be in the cloud as it could be on premises.
However, for CIP-007 R2 (patch management) and some other CIP requirements, it definitely does matter that the individual BES Cyber Assets are in scope. This is because there’s no way a cloud service provider could provide the evidence required to comply with CIP-007 R2. That’s because a system in the cloud is always split up among lots of different servers that might be in different data centers. In fact, they may move from one server to another from minute to minute.
Let’s suppose a NERC entity decides to implement a single medium impact BCS in the cloud, under today’s CIP requirements. Not thinking clearly, they ask the CSP to be sure to follow the requirements of CIP-007 R2. The CSP is sure to think their FedRAMP or ISO 27001 certification is all the evidence the entity will need – since both of those certifications require a much more rigorous patching regime than CIP-007 R2. So they will just provide the same FedRAMP evidence packet that they present to any cloud customer that asks for it.
However, if the entity presents the CSP’s evidence to the auditor, they’ll likely receive at least one Notice of Potential Violation (and perhaps many, each for the one BCS and the one requirement). The auditor will say, “It’s nice that the CSP has this certification, but that doesn’t fulfill CIP-007 R2.2” (let’s assume that R2.2 is the current focus of the audit). “However, please provide me evidence that you fulfilled the exact language of Part 2.2: ‘At least once every 35 calendar days, evaluate security patches for applicability that have been released since the last evaluation from the source or sources identified in Part 2.1.’ Moreover, I need this for every physical device on which any part of this BCS has resided during the three year audit period, even for just a few seconds.”
Of course, the number of physical devices in scope for Part 2.2, just for this one BES Cyber System, will be huge. The fact that they’re all covered by FedRAMP- or ISO 27001-compliant controls means nothing to the auditor. They need evidence (e.g. screenshots) that the steps required by Part 2.2 were performed for each device on which a part of the BCS resided for any amount of time during the three years, even if it’s not currently residing on that device.
Needless to say, gathering this evidence would require an astronomical amount of work – and that’s just for one BCS, for one part of one requirement. No CSP would ever promise to do this, if they knew this would be required of them. This is why it’s close to certain that no NERC entity with medium or high impact BCS has implemented even a single BCS in the cloud so far, even though the CIP requirements themselves don’t forbid use of the cloud.
So what needs to be done to solve this problem with the CIP requirements?
By accident, the way to solve this problem was pointed out by the CIP Modifications Standards Drafting Team in 2018 (although they weren’t even thinking about the cloud when they did this). They proposed that BES Cyber System be made the fundamental unit of CIP compliance and the terms Cyber Asset and BES Cyber Asset be retired. In other words, a BCS would no longer be just a combination of BES Cyber Assets. Instead, it would be a system that has a 15-minute impact on the grid (currently, the BCA must have a 15-minute impact). There would no longer be a place for individual devices in NERC CIP compliance, meaning the CSP would no longer need to track individual devices.
But the drafting team also realized in 2018 that, in order to make BCS the fundamental unit of compliance, all of the CIP requirements would need to be risk-based. There were already a number of risk-based requirements, like CIP-007 R3 (malware protection), CIP-010 R4 (Transient Cyber Assets) and CIP-011 R1 (information protection). But requirements like CIP-007 R2 (patch management) and CIP-010 R1 (configuration change management) are highly prescriptive, which is the opposite of risk-based. Prescriptive requirements would all need to be rewritten as risk-based.
When the drafting team announced they would need to make those changes, they expected to be greeted like conquering heroes, since everyone knew by that time that the most expensive requirements to comply with are the prescriptive ones like CIP-007 R2 (I was certainly ready to greet them as heroes, and I wrote at least four posts on their ideas).
However, the bigger utilities realized that, even though the prescriptive requirements were very expensive to comply with and required lots of meaningless work that had nothing to do with security, they had already made a huge investment in training, documentation, software, etc. for the existing requirements. They would have to throw all that away to start complying with the new risk based requirements. They dug in their heels, and that was the end of the drafting team’s idea for the time being.
However, that SDT’s ideas were revived last year and included in a new Standards Authorization Request (SAR) designed to finally fix the cloud problem. That SAR was approved by the NERC Standards Committee last December. Once again, the SAR makes BES Cyber System the fundamental unit of CIP compliance, but it does it just for cloud-based systems. If an entity doesn’t want to put any systems in the cloud, nothing will change. They’ll follow the same CIP requirements that they always have. Given this “two-track” approach, it seems unlikely that the same objections will be raised now that were raised in 2018.
When is it likely that the changes to CIP, that will allow all types of systems in scope for CIP to be implemented in the cloud, will be in place?
The SAR just allows the drafting team to be assembled. That won’t happen until the summer, and then the process of drafting the new standards will begin. Since these changes will be more thoroughgoing than previous CIP changes, and since there may also need to be changes to the NERC Rules of Procedure, I think it could easily take six years before the new standards are approved by NERC and FERC and ready to be implemented. There’s a legitimate question whether we have that much time; I don’t think we do, and I’m not alone. But that’s another discussion.
