There is certainly a difference between how cyber security software performs in a testing environment and how it performs in the real world. The many variables in real world usage patterns add an entirely new dimension to cyber security. Through my work at the Smart Grig Interoperability Panel (SGIP), I’ve come to believe that systems designers must consider the various means of inappropriate use or abuse capable by untrained users or nefarious individuals and provide appropriate testing for such usage at the outset.
When we install conformant and interoperable products in the smart grid and achieve interconnectivity and information flow only a few experts need to know how to use the software. The software is most often a transparent service to the users. But when we are implementing a cyber secure system, we have to implement processes that ensure that the users of the system (who typically are process-oriented individuals and not technical experts) are using it in the proper manner and not opening holes that breach security. Many breaches in systems occur because the user either did not configure them so they would be appropriately secure or because they used them outside of the environment they were intended to function.
Examples abound: Users did not select an appropriate password. Users did not use encryption on their laptop hard drive and it was stolen. That one should sound familiar, as it has occurred multiple times in organizations whose names you would immediately recognize. Users gave out their private security key or left a hardware token on their desk. Users added a device to their system which circumvented the organization’s security policy. Etc.
Therefore, interoperability and conformance testing cannot be content to simply focus on testing of message boundary exchanges and data structure syntax or even the presence of proper cyber related algorithms in the software under testing. Going with the examples above, tests might incorporate scenarios of data-at-rest encryption or dual-factor authentication or other product specific tests. While poor user interaction can never be totally predicted and fully addressed, it must be considered in developing interoperability and conformance testing.
So, we must view cyber security as an integral part of the interoperability and conformance testing – performing testing for all of them in a coordinated manner. We must have input from security professionals, both on how the software should be used as well as how it may be used in the real world. Only by employing such a unified approach can we have confidence that our testing methodology is appropriately focused. For more information, see the Smart Grid Testing and Testing Certification Committee wiki page. What are your experiences with these testing and usage challenges? Let us know.
Rik Drummond, CEO Drummond Group Inc
- An Accredited Test Lab and Certification Body by NAVLAP and ANSI
- Chair emeritus DoE’s Grid Wise Architecture Council
- Chair NIST Smart Grid Interoperability Panel’s Testing and Certification Committee