Past year I have spent internally within my team working on various CI/CD and Dev/Ops systems. Sometimes Chef, sometimes Ansible, often Open Shift and everything mostly tied together with Jenkins. There were three of us representing DEV, QE and OPS. We rebuilt our teams infrastructure from the ground up, and lot of the interesting will probably stay hidden behind our VPN and in our private repos.
We tried to put as much as possible to public repositories, but all of the configuration, our keys and secrets, and acompanying configuration scripts would stay hidden.
That is why I have decided to start writing down what we have learned. This year, our little tooling-team no longer exists, but I hope this will make the knowledge more accessible and that I would need to spend less time reading through the Jenkins javadocs :-)
You can configure everything through groovy scripts
This was probably the biggest advantage compared to our previous setups. All of the configuration is now in code. In our previous Jenkins instance, the configuration was mostly ad-hoc, manual, often depending on specific configuration our slaves. Changes to configuration was done with direct ssh to the slave through a shared account. Untangling the configuration was fun.
Now we use * credentials-binding plugin to share our centrally stored secrets across our jobs * config-file-provider to share the non-secret configuration * plain-credentials and ssh-credentials to store the secrets centrally on the master Jenkins
The interesting thing here is automating the update of the credentials. Currently, we rely on combination of groovy scripts and sh scripts running the jenkins-cli to run the groovy on Jenkins and so far it worked very well.
For example with this groovy file:
you could create a new credential just by running
Figuring out the groovy script can be tricky at times, but the upside is undeniable, and currently if we wanted to, we could configure a new instance of Jenkins under 30 minutes, including provisioning.
Using Jenkinsfiles and groovy pipelines is awesome
The ability to use Jenkinsfiles and groovy pipelines was one of the reasons we wanted to create new Jenkins infrastructure. The old Jenkins wouldn't be able to run these and when we were deciding wether to try to update the old or build a new one, we have decided to build a new one.
And pipelines did deliver. Groovy is much more flexible language than bash that we used previously for most of our automation.