Jenkins shared library is a powerful way for sharing Groovy code between multiple Jenkins pipelines. However, when many Jenkins pipelines, including mission-critical deployment pipelines, depend on such shared libraries, automated testing becomes necessary to prevent regressions whenever new changes are introduced into shared librariers. Despite its drawbacks, the third-party Pipeline Unit Testing framework satisfies some of automated testing needs. It would allow you to do mock execution of pipeline steps and checking for expected behaviors before actually running in Jenkins. However, documentation for this third-party framework is severely lacking (mentioned briefly here) and it is one of many reasons that unit testing for Jenkins shared libraries is usually an after-thought, instead of being integrated early. In this blog post, we will see how to do unit testing for Jenkins shared library with the Pipeline Unit Testing framework.
Testing Jenkins shared library
Example Groovy file
For this tutorial, we look at the following Groovy build wrapper as the example under test:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
After the shared library is set up properly, you can call the above Groovy build wrapper in Jenkinsfile as follows to use default parameters:
or you can set the parameters in the wrapper’s body as follows:
1 2 3
In the next section, we will look into automated testing of both use cases using JenkinsPipelineUnit.
To use JenkinsPipelineUnit, it is recommended to set up IntelliJ following this tutorial.
To test the above
buildWrapper.groovy using the Jenkins Pipeline Unit, you can start with a unit test for the second use case as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Unfortunately, when executing that unit test, it is very likely that you will get various errors that are not well-explained by JenkinsPipelineUnit documentation.
1 2 3 4 5 6
The short explanation is that the mock execution environment is not properly set up.
First, we need to call
setUp() from the base class BaseRegressionTest of JenkinsPipelineUnit to set up the mock execution environment.
In addition, since most Groovy scripts will have this statement
checkout scm, we need to mock the Jenkins global variable
scm, which represents the SCM state (e.g., Git commit) associated with the current Jenkinsfile.
The most simple way to mock it is to set it to empty state as follows:
We can also set it to a more meaningful value such as a Git branch as follows:
1 2 3 4 5 6 7 8
However, an empty
scm will usually suffice.
Besides Jenkins variables, we can also register different Jenkins steps/commands as follows:
After going through the setup steps above, you should have the following setup method like this:
1 2 3 4 5 6 7 8 9 10 11 12 13
Rerunning the above unit test will show the full stack of execution:
1 2 3 4 5 6 7 8 9 10 11 12
For automated detection of regression, we need to save the expected call stack above into a file into a location known to JenkinsPipelineUnit.
You can specify the location of such call stacks by overriding the field
callStackPath of BaseRegressionTest in
The file name should follow the convention
subname is specified by
testNonRegression method in each test case.
Then, you can update the above test case to perform regression check as follows:
1 2 3 4 5 6 7 8 9 10
In this example, the above call stack should be saved into
DemoTest_configured.txt file at the location specified by
Similarly, you can also have another unit test for the other use case of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Any change in
buildWrapper.groovy will be detected as test failures, as shown in the screen shot below.
In IntelliJ, we can click on Click to see difference link to compare the actual call stack versus the expected one that was saved in the text file.
You can also use PipelineUnitTests to test Jenkinsfile.
In most cases, testing Jenkinsfile will be similar to testing Groovy files in
vars folder, as explained above, since they are quite similar.
1 2 3 4 5 6 7 8 9 10 11
The process is very similar: you need to mock out some global variables and functions corresponding to Jenkins pipeline steps.
You will need to
printCallStack to obtain the expected output and save it into some text file.
Then, you can use
testNonRegression for automated verification of no-regression in Jenkinsfile.
This test class shows an example of testing Jenkinsfile using PipelineUnitTests.
Note that, unlike Groovy files in
vars folder, Jenkinsfiles are regularly updated and usually NOT depended/used by any other codes.
Therefore, automated tests for Jenkinsfile are not very common because of the cost/effort required.