The DoD has established an over-arching process to guide the Test and Evaluation community in assessing cybersecurity and resilience in weapon systems. The DoD publication Cybersecurity Test and Evaluation Guidebook outlines the process, identifies activities to be performed, and calls out documents to be produced during the system development process. While developed for the acquisition community as guidance in the procurement of new systems, the Guidebook and the processes described are applicable to and valuable for system development, modernization, and cyber awareness assessment at any stage in the system lifecycle.
KBSI has constructed an IDEF-0 model of the process described in the Guidebook to assist with understanding the complex task of performing cybersecurity within the Test and Evaluation community. The IDEF-0 model of activities to which this table refers can be found here. We encourage other providers of cybersecurity tooling to use this model as a reference to identify where and how their tools can be used to support the test and evaluation process.
The table matrix below describes “How and Where” the ASSURANT™ Suite capabilities fits into the Guidebook’s process to aid the Test and Evaluation community cybersecurity understanding and evaluations. As the ASSURANT™ Suite matures and is enhanced, this table will be updated to reflect new capabilities.
The IDEF-0 model of activities to which this table refers can be found here.
Table Key:
- [P] performs the function
- [S] supports (partially implements) the function)
- [I] informs (manages data relevant to) the function
- [C] captures and manages data from the function
- [U] utilizes data from the function
| NODE | DoD CT&E | ASSURANT Suite™ Capability |
| A0 | Assess Cyber Security Risk | |
| A1 | ~Phase 1: Understand Cyber Security Requirements | |
| A11 | ~~Compile Cybersecurity Requirements and Security Resources | |
| A111 | ~~~Examine Cybersecurity Standards | [S] Contains catalog of compliance standards |
| A112 | ~~~Examine Operational Resilience Requirements | [S] User identification of critical operational components |
| A113 | ~~~Examine System Cyber Survivability Requirements | [S] Model of system is effective reference |
| A12 | ~~Prepare for Phase 3 & 4 DT&E | |
| A121 | ~~~Develop the Initial DEF | |
| A1211 | ~~~~Define Security Capabilities | [S] Supports identification of mitigations required |
| A1212 | ~~~~Determine Evaluation Data Needed | |
| A1213 | ~~~~Determine Test Activities Needed |
[S] User Idenitfication of key terrain vulnerabilities [S] Identifies potential vulnerabilities based on model parameters |
| A1214 | ~~~~Incorporate Test Activities int Test Events and Document | |
| A122 | ~~~Identify Supporting Cybersecurity T&E Resources | [S] Generated documentation supports justification of T&E resource needs |
| A123 | ~~~Develop the Initial OT Evaluation Framework | |
| A124 | ~~~Align RMF Artifacts with the TEMP | |
| A125 | ~~~Align DCO Activities to Support the RMF | |
| A126 | ~~~Plan and Schedule MBCRA | [S] User identification of cybersecurity threats and impacts |
| A13 | ~~Develop Cyber Security T&E Strategy | |
| A2 | ~Phase 2: Characterize Attack Surface | |
| A21 | ~~Identify the Cyber-attack surface | |
| A211 | ~~~Examine System Architecture, Components, and Data Flows | |
| A2111 | ~~~~Identify System Components and Interaction Entities | [P] System model Identifies system components and interaction entities |
| A2112 | ~~~~Create Attack Surface List |
[P] User identification of system boundary [S] User identification and characterization of access points |
| A2113 | ~~~~Identify Key Terrain |
[S] User identification of system boundary and access point defines attack surface [S] User-established access point (EP) properties support identification of mission-critical points [S] User designation of mission-critical elements |
| A212 | ~~~Analyze and Decompose System Mission | |
| A213 | ~~~Map Mission Dependencies |
[S] System model identifies system components, interconnections, and data flows [S] System model identifies attack surface and access-point parameters |
| A214 | ~~~Examine Roles and Responsibilities | |
| A22 | ~~Analyze the Attack Surface |
[S] System model identifies attack surface and access-point parameters [S] User identification of threat and impact |
| A221 | ~~~Characterize the Cyber Threat | [S] User model of threat (capability, intention, motivation) and threat actors |
| A222 | ~~~Select a Cyber Kill Chain |
[S] System model identifies attack surface, system components, interconnections, and data flows [S] Attack Path Analysis identifies attack-path risks [S] Threat and threat-actor characterixation support threat analysis |
| A223 | ~~~Examine Cyber Effects on System and Mission |
[S] System model identifies attack surface, vulnerabilities, threats, and impacts [S] Attack Path identification and analysis support cyber-effects estimates [S] Threat model characterizes threats and threat actors |
| A224 | ~~~Perform or Update MBCRA | [S] System model, threat model, damage model (vulnerabilities and impacts) support planning and execution of MBCRA exersizes, including 'red team' preparation |
| A23 | ~~Document Results and Update Test Planning and Artifacts | [S] Document generator has access to all model data and can utilize generic or purpose-built report templates |
| A231 | ~~~Document Results of Cyber-attack Surface Analysis | [S] Document generator has access to all model data and can utilize generic or purpose-built report templates |
| A232 | ~~~Develop Threat Vignettes | [S] Threat model and attack paths support scripting of attack vignettes |
| A24 | ~~Preprare for Phase 3 and Phase 4 Cybersecurity DT&E events | |
| A241 | ~~~Formulate Test Strategy | |
| A242 | ~~~Schedule Test Event | |
| A3 | ~Phase 3: Identify Cybersecurity Vulnerabilities | |
| A31 | ~~Plan CVI Test Activities | |
| A311 | ~~~Develop Cybersecurity Test Objectives |
[S] System model identifies attack surface, vulnerabilities, and impacts [S] Attack path and data path identification support test objective definition and justification |
| A312 | ~~~Plan and Schedule Test Events | |
| A3121 | ~~~~Plan Test Events | |
| A31211 | ~~~~~Plan System Cyber Survivability Testing | [S] System model identifies attack surface, critical components, and attack impacts |
| A31212 | ~~~~~Plan Security Standards Testing | [S] Contains catalog of compliance standards |
| A31213 | ~~~~~Plan Operational Resilience Testing | [S] System model Identifies critical operational components (user-defined) |
| A31214 | ~~~~~Plan Integrated System Testing | |
| A3122 | ~~~~Plan Cyber Test Infrastructure | |
| A32 | ~~Conduct DVI Events and Document Results | |
| A321 | ~~~Obtain CVI Test Results | |
| A322 | ~~~Evaluate Cybersecurity |
[S] System model, threat model, and damage model provide effective reference for evaluations [S] Computes risk scores under multiple definitions |
| A323 | ~~~Update MBCRA |
[S] System model, threat model, damage model (vulnerabilities and impacts) support planning and execution of MBCRA exersizes, including 'red team' preparation [S] identifies potential mitigations [S] System model identifies mitigations (user selected or defined) and mitigation properties [S] Provides support for optimized selection of mitigations, trading off implementation cost, time, and effectiveness |
| A33 | ~~Document CVI Test Results | |
| A34 | ~~Prepare for Phase 4 Cybersecurity T&E Evenets | |
| A4 | ~Phase 4: Test & Evaluate Adversarial Impact on Critial Functions | |
| A41 | ~~Update Cyber Threat Assessment and Kill Chain Analysis | |
| A411 | ~~~Update Threat Assessment | [S] Threat model identifies threats and threat actors (capability, intention, motivation) |
| A412 | ~~~Update Kill Chain Analysis |
[S] System model identifies attack surface, system components, interconnections, and data flows [S] Attack Path Analysis identifies attack-path risks [S] Threat and threat-actor characterixation support threat analysis |
| A42 | ~~Plan Adversarial DT&E | |
| A421 | ~~~Develop Test Objective and Metrics | |
| A4211 | ~~~~Develop Test Objectives | |
| A42111 | ~~~~~Develop Operational Resilience Test Objectives |
[S] System model identifies attack surface, vulnerabilities, and impacts [S] Attack path and data path identification support test objective definition and justification |
| A42112 | ~~~~~Develop System Cyber Survivability Test Objectives |
[S] System model identifies attack surface, vulnerabilities, and impacts [S] Attack path and data path identification support test objective definition and justification |
| A42113 | ~~~~~Develop Security Standards Test Objectives | [S] Contains catalog of compliance standards |
| A4212 | ~~~~Integrate ACD Results for CVPA Testing | |
| A4213 | ~~~~Define Test Metrics | |
| A422 | ~~~Define Process and Test Cases | |
| A4221 | ~~~~Identify Resources | |
| A4222 | ~~~~Develop ROE | |
| A4223 | ~~~~Plan Integrated Test | [S] System model, threat model, and damage model provide effective reference for planning |
| A4224 | ~~~~Document Test Plans | |
| A423 | ~~~Finalize Preparation of Test Infrastructure | |
| A424 | ~~~Conduct TRR | [S] System model, threat model, and damage model provide effective reference for review |
| A43 | ~~Conduct ACD and Document Results | |
| A431 | ~~~Perform ACD Events | |
| A432 | ~~~Obtain ACD Reports | |
| A433 | ~~~Evaluate Cybersecurity (ACD) | [U] Facilitates update of threats, vulnerabilities, targets, impacts, which ensures currency of system model and encourages continuous re-assessment of cyber-risk status |
| A5 | ~Phase 5: Assess Vulnerabilities and Penetration | |
| A51 | ~~Plan CVPA | |
| A52 | ~~Coordinate with OTA | |
| A53 | ~~Execute CVPA & Document Results | [U] Facilitates update of threats, vulnerabilities, targets, impacts, which ensures currency of system model and encourages continuous re-assessment of cyber-risk status |
| A6 | ~Phase 6: Assess Adversary Immpact on Missions | |
| A61 | ~~Plan Adversarial Assessment (AA) | [S] System model, threat model, and damage model provide effective reference for planning |
| A62 | ~~Coordinate with the OTA Team | |
| A63 | ~~Execute AA and Document Results | [U] Facilitates update of threats, vulnerabilities, targets, impacts, which ensures currency of system model and encourages continuous re-assessment of cyber-risk status |
| A7 | ~Establish the Cybersecurity Working Group (CyWG) |
