Set up Jenkins for groovy script configuration

Why would you want to do this?

When I started working on new Jenkins configuration with my colleagues, high on our list of priorities was having reproducible configuration for our Jenkins.

Previously we maintained everything by hand, mostly through the web-based configuration, saved the resulting xmls, and hoped, that if something bad happens, we would be able to restore xml on new Jenkins instance and fix any occurring problems manually.

As we realized later, Jenkins is surprisingly configurable through it's embedded Groovy console. That let us to our plan, where we would write Groovy scripts for configuring various aspects of Jenkins and then tie all of them together with an overarching configuration shell-script.

For this to work, we needed to enable remoting over CLI. This guide will show you how.

Should you want to do this?

If you ever tried to do this before, you might have noticed the warnings, that enabling remoting leads to security vulnerability. You are practically giving access to all of the Jenkins internals to anybody with your key/password and you will be greeted with

"Allowing Jenkins CLI to work in -remoting mode is considered dangerous and usually unnecessary. You are advised to disable this mode."

and big blue button to disable remoting every time you go to "Manage Jenkins" through the web-console.

So, before you bring this into production maybe double-check if there is better way. Jenkins supports access over ssh protocol as well, even though in more limited fashion. Unfortunately, I haven't been able to find a way to distinguish the Jenkins CLI commands that would require remoting and those that would be fine with going through ssh/http.

In future I should look into the alternatives of directly running Groovy scripts.

For now, we really liked the flexibility that remoting gives us so we went all in.

How would you do this?

This is a good place to link the official documentation. Briefly, it could be summarized in these steps:

  • add your ssh public key to your account
  • enable TCP port for JNLP agents
  • enable remoting for CLI
  • download the client and run it with Java

After you are done with these four steps you should be able to run all of the CLI commands against your Jenkins.

Add your ssh public key to your account

In you account settings, found at https://$JENKINS_URL/user/$USERNAME/configure you should set your ssh public key.


If you are not familiar with setting up ssh keys, or you would like to generate new one, but keep forgetting how did you do that last time (as happens to me roughly once a month), you can consult a great guide on Github that explains the ssh generation step-by-step.

Enable TCP port for JNLP agents

The preferred option for connecting for CLI is JNLP. In theory, if this isn't enabled, the CLI binary should fall back to http. In practice, for certain commands, such as the groovy command we wanted to be using, you'd receive error about JNLP not being available.

To enable it, go to Manage Jenkins > Configure Global Security and choose "Fixed" or "Random" under TCP port for JNLP agents.


I would advise to set a specific port, so that you can enable the port in your firewall.

Enable remoting for CLI

Before you can use groovy scripts you need to enable remoting over CLI by checking the "Enable CLI over Remoting" in Manage Jenkins > Configure Global Security. As I mentioned, Jenkins will advise you against this, unfortunately, to take advantage of the groovy scripting through Jenkins CLI, this really needs to be enabled.


Download the cli client and run it

You can find the download link directly in your Jenkins instance, through Manage Jenkins > Jenkins CLI, you should even be able to download it directly from https://$JENKINS_URL/jnlpJars/jenkins-cli.jar. To execute it, you will need java runtime present.

To test out the setup, lets use the example you'd find if you wanted to try groovy-scripting from the web-console. The example there could be translated to a file plugin_list.groovy like this:

You should now be able to execute the file by running:

java -jar  jenkins-cli.jar -remoting -s https://$JENKINS_URL groovy plugin_list.groovy

and see something along the lines:

[Plugin:ssh-credentials, Plugin:workflow-job, Plugin:pipeline-github-lib, Plugin:workflow-basic-steps, Plugin:ws-cleanup, Plugin:pipeline-rest-api, Plugin:pipeline-model-api, Plugin:handlebars, Plugin:ace-editor]

To conclude

Now you have unlocked probably most versatile way to configure Jenkins. As they say, with great power comes great responsibility (alongside possible security vulnerabilities), but we really like the power! Most of the scripts I will be describing in other posts utilize groovy-scripts for configuration in one way or other, so unless I find a better way to do this sort of configuration, cli over remoting and groovy scripts are there to stay in our Jenkins configuration.

If you liked this, and want to ask anything, (or didn't and want to tell me how to do something better), feel free to write me an email to adam at!

Configuring secrets in Jenkins master from shell

To automatically configure secrets on Jenkins

first, a disclaimer

Just to make this upfront, I have tested this on Jenkins 2.89.3, with Credentials Binding plugin 1.14. I think I should make myself a CI/CD to test these snippets one day, because nobody likes a an out of date how-to, especially if that person will be me, in a year, trying to quickly solve a Jenkins configuration problem.

I assume that you have the account setup for cli usage, with alias for jenkins-cli in your shell. Second, there might be a better way. For example, if you are using Jenkins just to run your ansible-scripts, you could use ansible-valut, if you are relying on Kubernetes, you could configure secrets through kubectl.

In my experience, storing secrets in Jenkins is completely acceptable. The secrets are encrypted on rest. When you log them to console, Jenkins will transparently display ** instead of them. There is support for various types of them:

  • Plain text
  • Username/Password
  • SSH private key
  • AWS credentials

and more. Of course you could configure these through the management UI in the credentials section, but we didn't want to do that. We want to be able to have our credentials configured from code.

Second disclaimer, you might want to investigate if you want to have your credential scopes in Jenkins. We wave all of our credentials in the global scope, available of all jobs to use. We don't mind, because with our setup we would often have several distinct Jenkins instances, housing various secrets. For example Jenkins that is publicly accessible only has AWS credentials to push build artifacts to S3 bucket, but other instance, behind VPN, might have AWS credentials for EC2 as well, and we might run pipelines that spawn/update our staging infrastructure from there.

create a groovy script with the payload

For example with this groovy file, I can add credentials containing username and password:

There are only two lines really specific to plain text credentials. You can replace i.e.:

secret = new UsernamePasswordCredentialsImpl( //...

with other credential definitions. For example for plain-text credential (that has no username):

import org.jenkinsci.plugins.plaincredentials.impl.*
import hudson.util.Secret
def secret = new StringCredentialsImpl(CredentialsScope.GLOBAL,credId,description, Secret.fromString(password))

For ssh credentials:

import com.cloudbees.jenkins.plugins.sshcredentials.impl.*
import hudson.plugins.sshslaves.*;
def username = "HARDCODED"
def privateKeyStr = "-----BEGIN RSA PRIVATE KEY-----\nMII..."
def privateKeySource = new BasicSSHUserPrivateKey.DirectEntryPrivateKeySource(privateKeyStr)
def secret = new BasicSSHUserPrivateKey(

In case of ssh credentials, we usually hardcode the private key string in the groovy-file itself, because we encountered some problems where escaping mechanisms of shell seem to have collided with escaping mechanisms of groovy script transported over http. This would get even trickier if you'd needed to create a credential that is binary.

For AWS credentials, if we supply the access key as username and secret key as password:

import com.cloudbees.jenkins.plugins.awscredentials.*
def secret = new AWSCredentialsImpl(

execute the script with jenkins-cli

To create the credentials (in this case, I am assuming username password), just run the jenkins-cli groovy command. I have alias jenkins-cli=java -jar /opt/jenkins-cli.jar, and ssh key configured on my Jenkins master which means that I would run:

use from a Jenkinsfile

When I use these in my scripts, pipelines and Jenkinsfiles, I have to admit, that I usually generate the actual code with snippet-generator. This makes my work much easier especially if I need to work with credential I haven't worked with, or with several credentials at once. You should be able to find the generator on https://$JENKINSURL/pipeline-syntax/


If you'd then tried out a pipeline script such as this, you should be able to use the password injected in the environment variable.

node {
  withCredentials([string(credentialsId: 'example_secret, variable: 'TOKEN')]) {
    print "The secret is ${env.TOKEN}"

Beware, you shouldn't see the password itself if you print it, and this should give you some level of protection, i.e if you'd want to have your job-logs public (and we did want that for our opensource projects).

If you liked this, and want to ask anything, (or didn't and want to tell me how to do something better), feel free to write me an email to adam at!

The Best Things that I have found out about Jenkins in 2017

My Jenkins journey in 2017

Past year I have spent internally within my team working on various CI/CD and Dev/Ops systems. Sometimes Chef, sometimes Ansible, often Open Shift and everything mostly tied together with Jenkins. There were four of us representing DEV, QE and OPS. We rebuilt our teams infrastructure from the ground up, and lot of the interesting will probably stay hidden behind our VPN and in our private repos.

We tried to put as much as possible to public repositories, but all of the configuration, our keys and secrets, and accompanying configuration scripts would stay hidden.

That is why I have decided to start writing down what we have learned. This year, our little tooling-team no longer exists, but I hope this will make the knowledge more accessible and that I would need to spend less time reading through the Jenkins javadocs :-)

But before I begin, first I need to acknowledge that I wouldn't learn all of this, if not for hard work of Mike Nairn, Gerard Ryan and Paul McCarthy, three people who definitely did more than 3/4 of work in our team!

You can configure everything through groovy scripts

This was probably the biggest advantage compared to our previous setups. All of the configuration is now in code. In our previous Jenkins instance, the configuration was mostly ad-hoc, manual, often depending on specific configuration our slaves. Changes to configuration was done with direct ssh to the slave through a shared account. Untangling the configuration was fun.

Now we use

  • credentials-binding plugin to share our centrally stored secrets across our jobs
  • config-file-provider to share the non-secret configuration
  • plain-credentials and ssh-credentials to store the secrets centrally on the master Jenkins

The interesting thing here is automating the update of the credentials. Currently, we rely on combination of groovy scripts and sh scripts running the jenkins-cli to run the groovy on Jenkins and so far it worked very well.

Figuring out the groovy script can be tricky at times, but the upside is undeniable, and currently if we wanted to, we could configure a new instance of Jenkins under 30 minutes, including provisioning.

Using Jenkinsfiles and groovy pipelines is awesome

The ability to use Jenkinsfiles and groovy pipelines was one of the reasons we wanted to create new Jenkins infrastructure. The old Jenkins wouldn't be able to run these and when we were deciding whether to try to update the old or build a new one, we have decided to build a new one.

And pipelines did deliver. Groovy is much more flexible language than bash that we used previously for most of our automation. We managed to automate most of our release process and large parts of our deployment process. We don't think we would have managed that without groovy, as during the release process we need to process, build and tag over 60 repositories.

Another nice thing that emerged over the year is better support for groovy itself. Running pipelines should be resilient to things like shutdowns and restarts of Jenkins. Unfortunately this meant that for long time, you couldn't use standard groovy methods like collect or each, because the resulting code wouldn't be serializable, and you would need to use workaround with @NonCPS annotation.

Fortunately, since May 2017, most of these work, making for a much nicer environment to program in!

With Github Organization Folders you can create your own custom Travis

Even before we rebuilt our CI infrastructure, we have already been building most of our projects on Jenkins. To make the configuration more robust, we started experimenting with tools such as Jenkins Job Builder, because maintaining 60+ repositories in Github by hand was becoming unwieldy. Unfortunately Jenkins Job Builder scripts are quite complicated and cumbersome to use.

After we switched to new Jenkins, we could take advantage of the GitHub Organization Plugin, It automatically scans through all of our repositories in GitHub and creates a job for every repository that has a Jenkinsfile defining the CI pipeline for that repo. This lets us have our own infrastructure to provide something like Travis or CircleCI to our developers, while having much better control over the test environment.

Jenkins Job Builder is still useful

I have mentioned that we have previously used Jenkins Job Builder for configuring build jobs for each of our repositories and that it was cumbersome. I could identify several reasons why it wasn't working well:

  • We needed all of our developers to be able to understand most of the job definitions
  • We needed to utilize templating, to map our job definitions to 60+ repositories
  • Definitions themselves were mostly bash inside of templated yaml configuration, which lead to problems with escaping special characters
  • All the jobs were updated at once

After we moved GitHub Organization Plugin:

  • Developers can be interested just in the Pipelines that are useful for themselves
  • Pipeline libraries are much better way for code-sharing than yaml templating
  • Groovy code is more readable than bash inside of yaml
  • If you need to update a job, you just change the Jenkinsfile in a single repo

So why are we still using Jenkins Job Builder? It turns out, that you can use Jenkinsfiles with Jenkins Job Builder, and this solves most of the problems we encountered with the weird combination of templated yaml and bash. We still needed a way to define some of our long-running pipeline jobs, that weren't tied to any repository in particular, and for this, because most of the heavy lifting is done by the groovy code in separate file, Job Builder shines.

Job definition is

  • a simple, non-templated yaml file
  • defines name
  • defines triggers
  • points to the relevant Jenkinsfile

This means we can have all of the pipeline definitions in a single repository, and there is relatively few of them.

Working with Jenkins can be enjoyable

The last thing I should probably mention, is that I have learned to enjoy working with Jenkins. Despite sometimes being arcane, fiddly and hard to debug, it has over time become my favorite among the various CI systems, and where other people might build a static blog like this with travis, I run my personal Jenkins instance now!