Previously, we discussed how building automation systems have become a soft target for cyberattack. This is due to the large number of intelligent devices, open network protocols and increased reliance on third-party service providers. We also recommended automated cybersecurity monitoring and management frameworks as best practices to build a defense in depth appropriate for protecting these complex systems.
In this article we will consider the roles that third-party vendors must play to improve the cybersecurity of building automation systems. As a quick reminder, building automation systems perform many functions in large smart buildings (> 100,000 sq/ft or 10,000 sq/m), including HVAC control, lighting control, fire control, as well as physical security such as badge access systems and video surveillance. Elevator and parking garage controls are also included. Supporting these systems involves a cast of engineers, systems integrators and repair technicians, often logging in remotely to diagnose and make adjustments through software.
The cybersecurity of building automation systems is both a technical and business challenge. The ownership structure of buildings and building systems can vary widely, impacting the strategy used to establish defense in depth. Broadly speaking, there are two models: owner occupied and tenant leased buildings. Securing an owner occupied building is often less complicated, as ownership rests with the enterprise itself. This allows IT to engage, monitor and manage the network (LAN and VLAN) while working with Facilities (OT). Tenant leased buildings, which are the vast majority of the market, are more difficult as there are often multiple tenants from different companies sharing common infrastructure. Lease agreements are also unlikely to assign cybersecurity liabilities, which means responsibility for defense is left unfunded. For these reasons it is essential to understand the ownership model and any lease contracts as you embark on designing a defense. Ideally, you will have the full cooperation of various stakeholders, though in reality landlords and vendors will often hesitate to engage, and you will need to apply leverage to encourage participation.
Buildings generally follow a life cycle which includes construction, lease negotiation and occupancy. In most leased buildings, tenants lack contractual control over the cybersecurity provided by the owner, even when it is the tenant who is most vulnerable. It is during the construction and lease phases, when owners and tenants are negotiating other legal terms, that cybersecurity will have the best chance for attention (but only with active pressure from the prospective tenants). The US Department of Energy ESCS working group has developed an interesting procurement specification which tenants may find useful when negotiating new leases or even when forcing owners to address shortcomings post-lease and during the occupancy phase. Another useful reference is IEC 62443, originally intended for industrial control and SCADA systems, this standard can also be used to negotiate and specify building automation cybersecurity requirements.
A fact that most experts agree upon is that secure installations begin with zones of control. This is doubly important in building management systems, as they are often extensive with thousands of devices distributed throughout the building on open protocol networks such as BACnet. Building systems represent a large attack surface, and it is essential to partition the systems using VLANS, firewalls and routers. The goal is to ensure that any one branch of the system is isolated from other adjacent branches and that building systems are separated from business systems. During building construction, it is much easier for integrators to install systems so that they are flat and open across the entire network. If they are not required by contract to establish zones of control, it is highly likely that they will not spend time and money to do it.
Another mistake integrators often make is to place all of the building control systems on a common VLAN. While this may isolate them collectively from the shared tenant network and business systems, it does not protect them from each other. Ideally, HVAC controls should be on one VLAN, Physical Security on another, Lighting on another and so forth. The concept of air gapped systems is a nice idea and would be considered an extreme case of zones of control. However, practically speaking, air gapping is almost impossible to achieve in building automation systems. Instead, it is better to think of the zones as islands which are secured from within but can be interconnected through managed checkpoints. The idea is not to prevent all data from flowing, but to explicitly limit traffic with network configuration and to use your cybersecurity monitoring systems as needed.
Remote access is a special case of zones of control because it is widely used in the building automation industry and has increased recently as a result of social distancing protocols. There have been many articles written on best practices for remote access to industrial control systems, including this handy paper from the US Department of Homeland Security. The building industry is wholly dependent on remote service providers, and remote connectivity is unavoidable. However, extreme care must be taken to secure all (VPN) external connections, vendor endpoints, user credentials and traffic.
Another best practice which is often overlooked is establishing a consistent smart device addressing scheme, smart devices attached to IP networks should always be configured using static addressing. However, system integrators often use dynamic addresses during construction because it is expedient and because they do not have access to any enterprise static addresses when they are installing the system. Though dynamic addressing is convenient during installation, it can lead to vulnerabilities and should be corrected before occupancy. It is also important to establish alias naming conventions which describe a device's function and unique location within the building, these names will make cybersecurity monitoring, patch management and physical troubleshooting much easier when the facility is occupied.
The systems integrator should also establish secure access policies for users and groups of users. Policies and groups should cover building owners, tenants and vendors. Users, privilege levels, user groups, strong passwords and password expirations must be configured to follow industry best practices. Privileges in particular require care, as vendors will sometimes default to super admin privileges when logging in to perform even simple tasks. Simultaneous login with a common username must also be prevented. All users, including vendors, should be required to establish credentials which are unique to each person accessing the system and adhere to the established password expiration policy. In owner occupied buildings, it should be a best practice to register users on the corporate domain and to use LDAP as a means to authenticate user domain privileges before allowing access to building management systems. The pros and cons of this approach should be discussed with IT and the major vendors before activation.
Regular review of hardware and software revision levels should be made with the system integrator and/or manufacturer. This should include a review of operating systems and host servers. Cybersecurity automation software can be used to capture and report this information, including known vulnerabilities, and these reports should be used as a basis for the review with the integrator and the manufacturer. During the review process, it is essential to stay abreast of the manufacturer’s support policy for the installed platform, as this status can change, leaving a system unpatchable.
Building automation systems are often in place for 20 or more years. Obsolescence makes systems more vulnerable to attack, and older systems may be impossible to patch. Manufacturers should be transparent, knowledgeable and willing to review the status of the system. They should be willing to discuss options for patching, upgrading and replacing aging systems. Manufacturers must be willing and able to present their development methods aimed at achieving secure systems, including making systems secure by design and shipped secure by default. It is essential that manufacturers test devices frequently and have security monitoring in place from development to manufacturing to the end installation. Vendors who deflect or obstruct security reviews should be leveraged to comply or be replaced.
As you can see, there is a complicated mix of business contracts, systems and vendors needed to build a defense in depth. This is one reason why automated cybersecurity monitoring is essential. The best defense will identify problems early and provide clear information for remediation. Additionally, you must also build a team of experts, employees and vendors, who can be deployed quickly and effectively to remedy serious incidents, should they occur. Unfortunately, today there are too many tenants who are caught unaware, and the results can be catastrophic, which is unfortunate because many of these problems can be prevented with a measure of common sense, diligence and automation.
If you have any questions or comments about this article, please let us know via this link. In our next post, we will discuss more details about applying automation to detect and prevent cyberattacks. Until then, please stay safe.
