This appendix defines three assessment methods that can be used to assess the CUI security requirements in NIST Special Publication 800-171: examine, interview, and test. Included in the definition of each assessment method are types of objects to which the method can be applied. The application of each method is described in terms of the attributes of depth and coverage, progressing from basic to focused to comprehensive. The attribute values correlate to the assurance requirements specified by the organization.
The depth attribute addresses the rigor and level of detail of the assessment. For the depth attribute, the focused attribute value includes and builds upon the assessment rigor and level of detail defined for the basic attribute value; the comprehensive attribute value includes and builds upon the assessment rigor and level of detail defined for the focused attribute value.
The coverage attribute addresses the scope or breadth of the assessment. For the coverage attribute, the focused attribute value includes and builds upon the number and type of assessment objects defined for the basic attribute value; the comprehensive attribute value includes and builds upon the number and type of assessment objects defined for the focused attribute value.
Tables D-1 through D-3 provide complete descriptions of the examine, interview, and test assessment methods. The use of bolded text in the assessment method description indicates the content that was added to and appears for the first time, in the description indicating greater rigor and level of detail for the attribute value.
TABLE D-1: EXAMINE ASSESSMENT METHOD
| Method | EXAMINE The process of checking, inspecting, reviewing, observing, studying, or analyzing one or more assessment objects to facilitate understanding, achieve clarification, or obtain evidence. The results are used to support the determination of security safeguard existence, functionality, correctness, completeness, and potential for improvement over time. |
|
|---|---|---|
| Objects | Specifications | Examples: policies, plans, procedures, system requirements, designs. |
| Mechanisms | Examples: functionality implemented in hardware, software, firmware. | |
| Activities | Examples: system operations, administration, management, exercises. | |
| Attributes | Depth | Addresses the rigor of and level of detail in the examination process. |
| Basic | Examination that consists of high-level reviews, checks, observations, or inspections of the assessment object. This type of examination is conducted using a limited body of evidence or documentation. Examples include: functional-level descriptions for mechanisms; high-level process descriptions for activities; and documents for specifications. Basic examinations provide a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors. | |
| Focused | Examination that consists of high-level reviews, checks, observations, or inspections and more in-depth studies and analyses of the assessment object. This type of examination is conducted using a substantial body of evidence or documentation. Examples include: functional-level descriptions and where appropriate and available, high-level design information for mechanisms; high-level process descriptions and implementation procedures for activities; and documents and related documents for specifications. Focused examinations provide a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors and whether there are increased grounds for confidence that the safeguards are implemented correctly and operating as intended. | |
| Comprehensive | Examination that consists of high-level reviews, checks, observations, or inspections and more in-depth, detailed, and thorough studies and analyses of the assessment object. This type of examination is conducted using an extensive body of evidence or documentation. Examples include: functional-level descriptions and where appropriate and available, high- level design information, low-level design information, and implementation information for mechanisms; high-level process descriptions and detailed implementation procedures for activities; and documents and related documents for specifications.10 Comprehensive examinations provide a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors and whether there are further increased grounds for confidence that the safeguards are implemented correctly and operating as intended on an ongoing and consistent basis, and that there is support for continuous improvement in the effectiveness of the safeguards. | |
| Coverage | Addresses the scope or breadth of the examination process and includes the types of assessment objects to be examined; the number of objects to be examined by type; and specific objects to be examined.11 | |
| Basic | Examination that uses a representative sample of assessment objects (by type and number within type) to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors. | |
| Focused | Examination that uses a representative sample of assessment objects (by type and number within type) and other specific assessment objects deemed particularly important to achieving the assessment objective to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors and whether there are increased grounds for confidence that the safeguards are implemented correctly and operating as intended. | |
| Comprehensive | Examination that uses a sufficiently large sample of assessment objects (by type and number within type) and other specific assessment objects deemed particularly important to achieving the assessment objective to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors and whether there are further increased grounds for confidence that the safeguards are implemented correctly and operating as intended on an ongoing and consistent basis, and that there is support for continuous improvement in the effectiveness of the safeguards. | |
| DISCUSSION Typical assessor actions may include, for example: reviewing information security policies, plans, and procedures; analyzing system design documentation and interface specifications; observing system backup operations; reviewing training records; reviewing audit records; observing incident response activities; studying technical manuals and user/administrator guides; checking, studying, or observing the operation of an information technology mechanism in the system hardware or software; or checking, studying, or observing physical security measures related to the operation of a system. |
10 While additional documentation is likely for mechanisms when moving from basic to focused to comprehensive examinations, the documentation associated with specifications and activities may be the same or similar for focused and comprehensive examinations, with the rigor of the examinations of these documents being increased at the comprehensive level.
11 The organization, considering a variety of factors (e.g., available resources, importance of the assessment, the organization’s overall assessment goals and objectives), confers with assessors and provides direction on the type, number, and specific objects to be examined for the attribute value described.
TABLE D-2: INTERVIEW ASSESSMENT METHOD
| Method | INTERVIEW The process of conducting discussions with individuals or groups of individuals in an organization to facilitate understanding, achieve clarification, or lead to the location of evidence. The results are used to support the determination of security safeguard existence, functionality, correctness, completeness, and potential for improvement over time. |
|
|---|---|---|
| Objects | Individuals or Groups | Examples: Personnel with risk assessment responsibilities; personnel with information security responsibilities; system or network administrators; personnel with account management responsibilities. |
| Attributes | Depth | Addresses the rigor of and level of detail in the interview process. |
| Basic | Interview that consists of broad-based, high-level discussions with individuals or groups of individuals. This type of interview is conducted using a set of generalized, high-level questions. Basic interviews provide a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors. | |
| Focused | Interview that consists of broad-based, high-level discussions and more in- depth discussions in specific areas with individuals or groups of individuals. This type of interview is conducted using a set of generalized, high-level questions and more in-depth questions in specific areas where responses indicate a need for more in-depth investigation. Focused interviews provide a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors and whether there are increased grounds for confidence that the safeguards are implemented correctly and operating as intended. | |
| Comprehensive | Interview that consists of broad-based, high-level discussions and more in- depth, probing discussions in specific areas with individuals or groups of individuals. This type of interview is conducted using a set of generalized, high-level questions and more in-depth, probing questions in specific areas where responses indicate a need for more in-depth investigation. Comprehensive interviews provide a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors and whether there are further increased grounds for confidence that the safeguards are implemented correctly and operating as intended on an ongoing and consistent basis, and that there is support for continuous improvement in the effectiveness of the safeguards. |
|
| Coverage | Addresses the scope or breadth of the interview process and includes the types of individuals to be interviewed by role and responsibility; the number of individuals to be interviewed by type; and specific individuals to be interviewed.12 | |
| Basic | Interview that uses a representative sample of individuals in organizational roles to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors. | |
| Focused | Interview that uses a representative sample of individuals in organizational roles and other specific individuals deemed particularly important to achieving the assessment objective to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors and whether there are increased grounds for confidence that the safeguards are implemented correctly and operating as intended. | |
| Comprehensive | Interview that uses a sufficiently large sample of individuals in organizational roles and other specific individuals deemed particularly important to achieving the assessment objective to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors and whether there are further increased grounds for confidence that the safeguards are implemented correctly and operating as intended on an ongoing and consistent basis, and that there is support for continuous improvement in the effectiveness of the safeguards. | |
| DISCUSSION Typical assessor actions may include, for example, interviewing chief executive officers, chief information officers, senior information security officers, information owners, system and mission owners, system security officers, system security managers, personnel officers, human resource managers, network and system administrators, facilities managers, training officers, physical security officers, system operators, site managers, and users. |
12 The organization, considering a variety of factors (e.g., available resources, importance of the assessment, the organization’s overall assessment goals and objectives), confers with assessors and provides direction on the type, number, and specific individuals to be interviewed for the attribute value described.
TABLE D-3: TEST ASSESSMENT METHOD
| Method | TEST The process of exercising one or more assessment objects under specified conditions to compare actual with expected behavior. The results are used to support the determination of security safeguard existence, functionality, correctness, completeness, and potential for improvement over time.13 |
|
|---|---|---|
| Objects | Mechanisms | Examples: hardware, software, firmware. |
| Activities | Examples: system operations, administration, management; exercises. | |
| Attributes | Depth | Addresses the types of testing to be conducted. |
| Basic | Test methodology (also known as black box testing) that assumes no knowledge of the internal structure and implementation detail of the assessment object. This type of testing is conducted using a functional specification for mechanisms and a high-level process description for activities. Basic testing provides a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors. | |
| Focused | Test methodology (also known as gray box testing) that assumes some knowledge of the internal structure and implementation detail of the assessment object. This type of testing is conducted using a functional specification and limited system architectural information (e.g., high-level design) for mechanisms and a high-level process description and high-level description of integration into the operational environment for activities. Focused testing provides a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors and whether there are increased grounds for confidence that the safeguards are implemented correctly and operating as intended. | |
| Comprehensive | Test methodology (also known as white box testing) that assumes explicit and substantial knowledge of the internal structure and implementation detail of the assessment object. This type of testing is conducted using a functional specification, extensive system architectural information (e.g., high-level design, low-level design) and implementation representation (e.g., source code, schematics) for mechanisms and a high-level process description and detailed description of integration into the operational environment for activities. Comprehensive testing provides a level of understanding of the security safeguards necessary for determining whether the safeguards are implemented and free of obvious errors and whether there are further increased grounds for confidence that the safeguards are implemented correctly and operating as intended on an ongoing and consistent basis, and that there is support for continuous improvement in the effectiveness of the safeguards. | |
| Coverage | Addresses the scope or breadth of the testing process and includes the types of assessment objects to be tested; the number of objects to be tested by type; and specific objects to be tested. | |
| Basic | Testing that uses a representative sample of assessment objects by type and number within type, to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors. | |
| Focused | Testing that uses a representative sample of assessment objects by type and number within type, and other specific assessment objects deemed particularly important to achieving the assessment objective to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors and whether there are increased grounds for confidence that the safeguards are implemented correctly and operating as intended. | |
| Comprehensive | Testing that uses a sufficiently large sample of assessment objects by type and number within type, and other specific assessment objects deemed particularly important to achieving the assessment objective to provide a level of coverage necessary for determining whether the security safeguards are implemented and free of obvious errors and whether there are further increased grounds for confidence that the safeguards are implemented correctly and operating as intended on an ongoing and consistent basis, and that there is support for continuous improvement in the effectiveness of the safeguards. | |
| DISCUSSION Typical assessor actions may include, for example: testing access control, identification and authentication, and audit mechanisms; testing security configuration settings; testing physical access control devices; conducting penetration testing of key system components; testing system backup operations; testing incident response capability; and exercising vulnerability scanning capability. |
13 Testing is typically used to determine if mechanisms or activities meet a set of predefined specifications. Testing can also be performed to determine characteristics of a security or privacy control that are not commonly associated with predefined specifications, with an example of such testing being penetration testing.