In this post, we look into some troubleshooting tips when using independent Groovy scripts in Jenkins pipeline and how to work around those.

Cannot load a Groovy script in Declarative Pipeline

Problem: Loading Groovy methods from a file with load step does not work inside Declarative Pipeline step, as reported in this issue.

Workaround: There are a few work-arounds. The most straight-forward one is to use script step.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

steps {
    checkout scm
    withCredentials([
        [$class: 'StringBinding', credentialsId: 'nexusUserName', variable: 'nexusUserName'],
        [$class: 'StringBinding', credentialsId: 'nexusPassword', variable: 'nexusPassword']
    ]) {
        script {
            myScript = load 'jenkins/xml.groovy'
            String myFile = myScript.transformXml(settingsFile, env.nexusUserName, env.nexusPassword)
            sh "mvn -B -s ${myFile} clean compile"

            sh "rm ${myFile}"
        }
    }
}

You can also define Groovy methods from inside the Jenkinsfile.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

import groovy.xml.StreamingMarkupBuilder
import groovy.xml.XmlUtil

@NonCPS
def xmlTransform(txt, username, password) {

    def xmlRoot = new XmlSlurper(false, false).parseText(txt)
    echo 'Start tranforming XML'
    xmlRoot.servers.server.each { node ->
       node.username = username
       node.password = password
    }

    // TRICKY: FileWriter does NOT work
    def outWriter = new StringWriter()
    XmlUtil.serialize( xmlRoot, outWriter )
    return outWriter.toString()
}

...

   steps {
        checkout scm
        withCredentials([
            [$class: 'StringBinding', credentialsId: 'nexusUserName', variable: 'nexusUserName'],
            [$class: 'StringBinding', credentialsId: 'nexusPassword', variable: 'nexusPassword']
        ]) {
            script {
                myScript = load 'jenkins/xml.groovy'
                String myFile = xmlTransform(settingsFile, env.nexusUserName, env.nexusPassword)
                sh "mvn -B -s ${myFile} clean compile"

                sh "rm ${myFile}"
            }
        }
    }

For Declarative Pipeline, to reuse the code from a Groovy script, you must use Shared Libraries. Shared Libraries are not specific to Declarative; they were released some time ago and were seen in Scripted Pipeline. This blog post discusses an older mechanism for Shared Library. For the newer mechanism of importing library, please check out this blog post. Due to Declarative Pipeline’s lack of support for defining methods, Shared Libraries definitely take on a vital role for code-reuse in Jenkinsfile.

File reading and writing not supported

Java/Grooy reading and writing using “java.io.File” class is not directly supported.

1

def myFile = new File('/home/data/myfile.xml')

In fact, using that class in Jenkinsfile must go through “In-Process Script Approval” with this warning.

new java.io.File java.lang.String Approving this signature may introduce a security vulnerability! You are advised to deny it.

Even then, “java.io.File” will refer to files on the master (where Jenkins is running), not the current workspace on Jenkins slave (or slave container). As a result, it will report the following error even though the file is present in filesystem (relevant Stackoverflow) on slave:

1
2

java.io.FileNotFoundException: /home/data/myfile.xml (No such file or directory)
  at java.io.FileInputStream.open0(Native Method)

That also means related class such as FileWriter will NOT work as expected. It reports no error during execution but you will find no file since those files are created on Jenkins master.

Workaround:

• For file reading, use readFile step.
• For file writing, use writeFile step. However, Pipeline steps (such as writeFile) are NOT allowed in @NonCPS methods. For more complex file writing, you might want to export the file content as String and use the following code snippet:

1
2
3
4
5
6

String xmlFile = ...

// TRICKY: FileWriter does NOT work in xmlTransform
def mCommand = "cat >${settingsFile} <<EOF"
mCommand += "\n${xmlFile}\nEOF"
sh mCommand

In the code snippet above, we construct a here document-formatted command for writing multi-line string in mCommand before passing to sh step for executing.

1
2
3
4
5
6
7
8
9
10

$ cat >output.txt <<EOF
SELECT foo, bar FROM db
WHERE foo='baz'
More line from xmlFile
EOF

$ cat output.txt
SELECT foo, bar FROM db
WHERE foo='baz'
More line from xmlFile

Serialization errors

You often encounter this type of errors when using non-serialiable classes from Groovy/Java libraries.

1
2

java.io.NotSerializableException: org.codehaus.groovy.control.ErrorCollector
  at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)

1
2
3
4
5
6

java.lang.UnsupportedOperationException: Calling public static java.lang.Iterable 
org.codehaus.groovy.runtime.DefaultGroovyMethods.each(java.lang.Iterable,groovy.lang.Closure) on a
CPS-transformed closure is not yet supported (JENKINS-26481); 
encapsulate in a @NonCPS method, or use Java-style loops
  at org.jenkinsci.plugins.workflow.cps.GroovyClassLoaderWhitelist.checkJenkins26481
    (GroovyClassLoaderWhitelist.java:90)

There is also some known issue about JsonSlurper. These problems come from the fact that variables in Jenkins pipelines must be serializable. Since pipeline must survive a Jenkins restart, the state of the running program is periodically saved to disk for possible resume later. Any “live” objects such as a network connection is not serializble.

Workaround: Explicitly discard non-serializable objects or use @NonCPS methods.

Quoted from here: @NonCPS methods may safely use non-Serializable objects as local variables, though they should NOT accept nonserializable parameters or return or store nonserializable values. You may NOT call regular (CPS-transformed) methods, or Pipeline steps, from a @NonCPS method, so they are best used for performing some calculations before passing a summary back to the main script.

References

• Declarative syntax
• Shared Libraries for Declarative Pipeline
• Here document
  • Theory
  • Common usage from Stackoverflow
  • Heredoc with and without variable expansion