Skip to content

Page388

Assessing the Effectiveness of Software Security

Once the project is underway and software has been programmed, the next steps are testing the software, focusing on the confidentiality, integrity, and availability of the system, the application, and the data processed by the application. Special care must be given to the discovery of software vulnerabilities that could lead to data or system compromise. Finally, organizations need to be able to gauge the effectiveness of their software creation process, and identify ways to improve it.

Software Vulnerabilities

Programmers make mistakes: this has been true since the advent of computer programming. In Code Complete, Steve McConnell says, “experience suggests that there are 15–50 errors per 1000 lines of delivered code” [28]. One thousand lines of code are sometimes called a KLOC; “K” stands for thousand. Following a formal application maturity framework model can lower this number. Watts S. Humphrey, a Fellow at Carnegie Mellon University’s Software Engineering Institute, claims that organizations that follow the SEI Capability Maturity Model Integration (CMMI, see “Software Capability Maturity Model Integration (CMMI)” section below) can lower the number of errors to one in every KLOC [29].

Even one error per thousand lines of code can introduce large security risks, as our software becomes increasingly complex. Take Microsoft Windows, for example: “As a result, each new version of Windows carries the baggage of its past. As Windows has grown, the technical challenge has become increasingly daunting. Several thousand engineers have labored to build and test Windows Vista, a sprawling, complex software construction project with 50 million lines of code, or more than 40% larger than Windows XP” [30]. Note that Microsoft has not released the number of lines of code of its recent operating systems, including Windows 11 and Server 2022.

If the Microsoft Vista programmers made only one error per KLOC, then Vista has 50,000 errors. Large software projects highlight the need for robust and methodical software testing methodologies.

Types of Software Vulnerabilities

This section will briefly describe common application vulnerabilities. Please also refer to the “System Vulnerabilities, Threats, and Countermeasures” section of Chapter 4, Domain 3: Security Architecture and Engineering, for information regarding additional vulnerability types. An additional source of up-to-date vulnerabilities can be found at “CWE Top 25 Most Dangerous Software Weaknesses,” available at https://cwe.mitre.org/top25/archive/2021/2021_cwe_top25.html; the following summary is based on this list. CWE refers to Common Weakness Enumeration, a dictionary of software vulnerabilities by MITRE (see https://cwe.mitre.org/).

  • Out-of-bounds write (aka buffer overflow, occurs when a programmer does not perform variable bounds checking)
  • Cross-site scripting (Improper Neutralization of Input During Web Page Generation)
  • Out-of-bounds write (reading past the end of a buffer)
  • Improper Input Validation
  • OS Command injection
  • SQL Injection: manipulation of a back-end SQL server via a front-end web server
  • Cross-Site Request Forgery (CSRF)
  • Directory Path Traversal: escaping from the root of a web server (such as /var/www) into the regular file system by referencing directories such as “../../” [31]