Andrew Mobbs | Software Architect
Systems
The word “system” comes from the Greek σύστημα, a whole thing made of parts, or a composition. When we discuss the behaviour of systems, we’re discussing the emergent behaviour arising from the interaction of the many elements that the system is composed of and the interactions between the system and its environment.
Ensuring that systems only behave in the way the designer intends is a central aspect of security. A security-enforcing system will take the context of available information measured against policy to determine whether an operation should be permitted or denied. An attacker may try to manipulate or bypass various aspects of that policy enforcement to perform an operation that should be denied, or in the case of a denial-of-service, to deny an operation that should be permitted.
Thus, we need to engineer secure systems such that they only operate in the way the designer intends. With complex systems this becomes a hard problem as the system will be composed of many elements often developed by distributed teams and designers with interactions between elements of the system that are increasingly difficult to understand. To build secure systems, we need to build systems that we can understand, reason about and communicate that understanding precisely between collaborating designers.
Systems Engineering and software
Systems Engineering is the interdisciplinary approach to realizing systems that meet desired goals. It was developed over the 20th century to enable the successful realization of ever more complex engineering projects; from telephony to military, space and automotive domains.
Traditional engineering approaches have often been perceived by software practitioners as cumbersome and inflexible compared with the near infinite variability and rapid adaptability of software. The term “software engineer” is often deliberately muddled with “programmer”, “developer”, “coder” or the like with the activity often involving very little engineering. Complexity in software is handled through abstraction and layering, and all too often unintended interactions between system elements or the emergent side-effects of intended interactions are not considered.
Experienced and skilled programmers often avoid such unintended consequences through building a deep internalized understanding of the system they’re working with. Practices like test-driven development and tooling such as static analysis and fuzzing as well as traditional testing can help discover or avoid some problems. However, developing complex software systems with large teams is still notably a fallible, expensive and inconsistent exercise.
Safety-critical software systems such as avionics and some high assurance security-critical systems have always had strong engineering requirements. As many more software systems govern important aspects of life and are exposed to security risks by being connected to the internet, the same robust engineering approaches need to be applied.
Those factors mean the software industry needs to adopt better tools and processes for communicating about and understanding the systems they build to make them secure enough for use in applications with real-world consequences if the systems are misused.
Model Based Systems Engineering
Modern systems engineering has developed techniques and approaches that address many of the concerns around the inflexible and arduous processes that were previously inherent in the document-centric approaches. These techniques are encompassed in the “Model Based Systems Engineering” or MBSE.
A “model” is an abstraction of reality and “modelling” is the practice of creating and using models. Model Based System Engineering “is the formalized application of modelling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases” [1]. A full introduction to MBSE can be found on the INCOSE web site. [2]
By creating models accurately abstracting the complexity of the real-world system, we can reason about specific aspects of system behaviour in a consistent manner without losing important characteristics of the whole system.
MBSE is fully compatible with modern approaches to software architecture such as ISO 42010:2011. In software architecture, we often use models to describe the layers and interactions of software elements. What MBSE introduces is a strong consistency both internally within the model and between the model and the real system which allows for stronger reasoning based on the model alone. A designer can trust that the model and the system will behave in the same way.
MBSE and security engineering
Bruce Schneier argues “Good engineering involves thinking about how things can be made to work; the security mindset involves thinking about how things can be made to fail.[…] The lack of a security mindset explains a lot of bad security out there: voting machines, electronic payment cards, medical devices, ID cards, internet protocols. The designers are so busy making these systems work that they don't stop to notice how they might fail or be made to fail, and then how those failures might be exploited.” [3]
There is some truth to this; although it is a mischaracterization of good engineering to exclude unintended consequences of system behaviour, it’s true that the intended behaviour is often given far more weight than the “rainy day” scenarios in both design and implementation of systems.
To fully realize security benefits from MBSE in a software system the approach needs to be fully integrated into the whole system lifecycle. Views should be provided that explicitly considers the security engineering world of threats, controls, misuse cases, assets and so on. Additionally, appropriate security aspects should be brought into the general system engineering views to ensure they’re considered as an integral part of system design. MBSE will ensure the specialist and generalist views remain consistent.
MBSE improves communication between specialists and generalists by constraining the vocabulary used to describe characteristics and by providing clear definitions of the terms used for model elements and their relationship to other model elements. Using these definitions, specialist vocabulary can be learned quickly and related to already understood aspects of the system. By providing a framework of interdependent viewpoints for describing the system, MBSE allows specialists and generalists to collaborate on the same model of a system.
In conclusion, Model Based Systems Engineering helps us to design, build and operate complex systems that are more secure because we understand how they will behave both when used as intended and when misused by attackers.
References
[1] INCOSE, “INCOSE SE Vision 2020 (INCOSE-TP-2004-004-02),” 2007.
[2] INCOSE, “What is Model Based Systems Engineering?,” [Online]. Available: http://www.incoseonline.org.uk/Program_Files/Publications/zGuides_9.aspx?CatID=Publications.
[3] B. Schneier. [Online]. Available: https://www.schneier.com/blog/archives/2008/03/the_security_mi_1.html.