Automated Regression Testing
Regression tests ensure that code that was previously functioning correctly does not regress when changes are made. Having a comprehensive suite of unit level regression tests provides a safety net, making sure that code changes do not break existing functionality.
It is common to automate regression tests that are running. However, some automated regression testing tools make it possible to automate regression testing for multiple test stages:
Automatic regression test generation
Depending solely on system testing can be highly expensive and inefficient when it comes to modifying legacy code. On the other hand, creating a comprehensive set of unit tests manually from scratch may not be commercially feasible.
To solve this problem, automated test tools can be utilized to generate a suite of passing unit tests by parsing the source code and determining all possible paths through the code. The test suite can be generated based on a specified structural code coverage target.
These auto-generated tests can be used as a baseline safety net to detect regression bugs when code is altered. This reduces reliance on time-consuming and expensive system tests while being more thorough and precise in identifying the location of errors.
If you want to learn more about automatic test generation for safety-critical C code, check out Cantata.
Run regression tests automatically
Automating the process of re-running a set of regression tests that have passed can be achieved using a continuous integration solution like Jenkins® or Bamboo®.
By running regression tests automatically, it becomes easier to test each build. Frequent test runs help identify regression errors early on, making it easier to fix bugs. This is particularly helpful as code changes are still fresh in the developer’s mind, and the problem is less likely to have been compounded by further changes.
Automating regression test maintenance
Changes to the source code can sometimes require updating the regression tests. Maintaining tests can be costly if significant developer effort is needed to identify dependencies and update tests to ensure they are compatible with the modified code under test.
Much of this process can be automated. The Cantata Code Change Analyzer uses information from the last time tests were run to automate test updates. When there is more than one appropriate way to update tests, an automated test tool can offer developers options for resynchronizing tests with the code:
- The tool can suggest suitable updates to the test for each code change.
- It can provide guidance on the effects of accepting the suggestion.
- It can automatically refactor tests to implement the suggested update.
