Flight-Ready Software: Ensuring Reliability Through Testing

Levels of Testing in Software Development, Software testing is conducted at multiple levels to verify different aspects of the software application. Below are the key levels of software testing:

Figure1: Levels of software testing. Source: author, built with draw.io

1. Unit Testing

Unit testing is a software development process in which individual components of an application, known as units, are tested in isolation to ensure they function correctly. These units, often the smallest testable parts of the software, such as functions or methods, are scrutinized independently from the rest of the application. The primary objective of unit testing is to validate that each unit performs as intended, facilitating early detection of errors and simplifying code maintenance. By isolating each part of the program, unit testing provides a written contract that the code must satisfy, ensuring reliability and correctness in the software development process.

Unit testing is performed by writing test cases to verify individual functions or methods, executing them using frameworks like JUnit, PyTest, or Jest, and analyzing the results. If a test fails, developers fix the issue and re-run the tests to ensure correctness. Automated unit tests are often integrated into CI/CD (Continuous Integration/Continuous Deployment) pipelines to maintain code quality throughout development. This process helps detect errors early, improves maintainability, and ensures software reliability.

NASA’s Mars Climate Orbiter failure in 1999:

A famous real-world example that highlights the importance of unit testing is NASA’s Mars Climate Orbiter failure in 1999. The spacecraft, designed to study the Martian atmosphere, was lost due to a software error caused by inconsistent unit conversions. Specifically, one software component generated force values in pound-force, while another part of the system expected them in newton, the metric equivalent. This mismatch led to incorrect trajectory calculations, ultimately causing the spacecraft to enter Mars’ atmosphere at a dangerously low altitude and burn up. Proper unit testing could have detected this issue early by verifying the correctness of individual components responsible for calculations. By isolating and testing each function’s output, engineers would have identified the inconsistency before integration, preventing the mission’s catastrophic failure. This incident underscores how a seemingly small unit-level mistake can have significant real-world consequences, emphasizing the critical role of unit testing in software development.

2. Integration Testing

​Integration testing is a crucial phase in software testing where individual modules are combined and tested collectively to identify defects in their interactions and interfaces. This process ensures that integrated components function together as intended, revealing issues that may not surface during unit testing. By focusing on the communication between modules, integration testing validates the overall system’s cohesion and reliability.

Integration testing ensures that software modules work together correctly. It includes Top-Down Testing (starting from high-level modules using stubs), Bottom-Up Testing (beginning with low-level modules using drivers), and Sandwich Testing (a hybrid approach). Tools like Selenium (for web automation), Postman (for API testing), and JMeter (for performance testing) help streamline the process. By using these methods and tools, developers can detect integration issues early, improving software reliability.

3. System Testing

​System testing is a comprehensive evaluation of a fully integrated software system to ensure it meets specified requirements. This phase involves testing the complete application, including its functionality, performance, security, and usability, to verify that all components work together as intended. System testing is typically conducted after integration testing and before acceptance testing, serving as a critical step in identifying defects and ensuring the software’s readiness for deployment.

4. Acceptance Testing

Acceptance testing is a quality assurance process that evaluates whether a software application meets specified business requirements and user needs before its release. This final phase of testing involves validating the software’s functionality, performance, and usability to ensure it aligns with end-user expectations and is ready for deployment. Acceptance testing encompasses various types, including user acceptance testing (UAT), operational acceptance testing (OAT), and contract acceptance testing, each serving to confirm the software’s readiness from different perspectives. By conducting thorough acceptance testing, organizations can identify and address potential issues before the software reaches the production environment, thereby enhancing product quality and user satisfaction.

Table 1: Levels of Software Testing Overview

Financial Impact of Software Testing on Companies

Implementing an effective software testing process has a direct financial impact on businesses. While testing incurs costs, it ultimately saves companies from larger financial losses. Here’s how:

• Reduced Post-Deployment Fixes: Fixing software issues after deployment is costly. A robust testing process helps catch and resolve errors early.
• Improved Market Reputation: Reliable software leads to positive customer reviews, increasing sales and business growth.
• Avoidance of Legal Penalties: Compliance failures can lead to heavy fines and legal issues, making testing crucial in regulated industries.
• Reduced Downtime Costs: Bugs in critical business applications can result in downtime, leading to revenue losses. Thorough testing minimizes this risk.
• Increased Customer Retention: Poorly functioning software can drive customers away. A well-tested product enhances user satisfaction and loyalty.

On the other hand, as mentioned earlier, the testing process incurs costs for businesses and businesses must perform optimal testing to balance between software quality and cost. An article published in 2016 (6th International Conference on Information and Communication Technology) shows the relation between quality and number of testing.

Figure 2: Amount of testing vs Quality

This chart illustrates the relationship between the amount of testing and quality, cost, and over-testing. It is clearly that too little testing results in poor quality, whereas excessive testing leads to high costs and over-testing. The optimal testing range is likely somewhere in the middle, balancing cost-effectiveness and quality assurance.
This chart is from an academic publication and effectively illustrates the concept. However, in some projects, certain software requires maximum quality. In these cases, managers must prioritize quality over testing costs, ensuring the highest standards without compromise.

Given the financial implications of software testing, businesses must carefully structure their testing strategy. Before diving into the details of system testing, I want to mention one point that I think will help us see the big picture. Since testing is a crucial step it must be well structured. So a model like Software Testing Life Cycle (STLC) may help responsible individuals. In the article “Software Testing Techniques: A Literature Review” its described as shown in the figure 3:  

Figure 3: Software Testing Life Cycle (STLC). Source: author, built with draw.io

Let’s dive into more detail in System Testing.

So why do we perform system testing? and what are the key objectives of system testing?

1. Verify System Functionality – Ensure that all features work as intended.
2. Validate System Performance – Assess how the system behaves under different workloads.
3. Ensure Security Compliance – Identify vulnerabilities and ensure data protection.
4. Check Compatibility – Test system behavior across different devices, operating systems, and environments.
5. Detect System-Wide Defects – Find bugs that unit and integration testing might have missed.

Types of System Testing

System testing includes multiple subtypes, each focusing on different aspects of the software. During my research, I found classifications in many different reliable sources. I felt like all of these sources made the similar classification with different expressions. I wanted to use the classification shared in the career development section of the Indeed website in the article because of its understandable and clear structure.

Figure 4: Types of system Testing. Source: author, built with draw.io

Who Performs System Testing?

System testing is typically performed by QA (Quality Assurance) testers who evaluate the complete system’s functionality, performance, and security. They use manual and automated testing techniques to identify potential issues before the software moves to the acceptance testing phase.

When is System Testing Conducted?

System testing is conducted after integration testing is completed and before acceptance testing begins. It takes place once all modules and components have been successfully integrated and the system is ready for real-world validation. This ensures that the entire application functions correctly as a unified system before it moves to the final testing phase.

Tools Used for System Testing

Various tools are used for system testing to ensure software quality and performance. Selenium is commonly utilized for automated functional testing, allowing testers to verify system functionalities efficiently. JMeter is widely used for performance testing, helping evaluate how the system behaves under different loads. For API testing, Postman is a preferred tool, enabling testers to validate API interactions and responses. Additionally, LoadRunner is employed for load testing, assessing how the system performs under high user traffic and stress conditions. These tools collectively help ensure the software meets performance, functionality, and reliability standards before deployment.

Why is System Testing Important?

System testing ensures that the software functions correctly in a real-world environment, reducing the chances of post-deployment failures. It helps detect critical system-wide defects, improves system reliability, and ensures compliance with user and business expectations.

Conclusion

Software testing is a vital part of the development process, ensuring high-quality, secure, and reliable applications. Different levels of testing—Unit, Integration, System, and Acceptance—help verify various aspects of the software, reducing errors and enhancing functionality. Investing in thorough testing not only improves software quality but also saves companies from financial losses, enhances brand reputation, and ensures a superior user experience.

By prioritizing software testing, businesses can avoid costly mistakes and deliver software that meets both user expectations and industry standards.

Recommended Article: Waterfall vs. Agile Project Management: Key Differences, Techniques, and Best Practices

References:

• -Software Testing Techniques: A Literature Review-2016 6th International Conference on Information and Communication Technology for The Muslim World
• – Software Testing In A System Development Process: A Life Cycle Perspective: Journal of Systems Management, 41 (8), August 1990, 23-31
• –https://circleci.com/blog/unit-testing-vs-integration-testing/
• -https://www.browserstack.com/guide/integration-testing
• – https://www.softwaretestingmaterial.com/integration-testing/
• – https://www.browserstack.com/guide/acceptance-testing
• – https://www.geeksforgeeks.org/system-testing/
• – https://ca.indeed.com/career-advice/career-development/system-testing