Pipeline-as-code or defining the deployment pipeline through code rather than manual job creation through UI, provides tremendous benefits for teams automating builds and deployment infrastructure across their environments.
Source of image: https://jenkins.io/doc/book/pipeline/
Jenkins is a well-known open source continuous integration and continuous deployment automation tool. With the latest 2.0 release, Jenkins introduced the Workflow plugin that implements Pipeline-as-code. This plugin lets you define delivery pipelines using concise scripts which deal elegantly with jobs involving persistence and asynchrony.
The Pipeline-as-code’s script is also known as a Jenkinsfile.
Jenkinsfiles uses a domain specific language syntax based on the Groovy programming language. They are persistent files which can be checked in and version-controlled along with the rest of their project source code. This file can contain the complete set of encoded steps (steps, nodes, and stages) necessary to define the entire application life-cycle, becoming the intersecting point between development and operations.
Missing piece of the puzzle
One of the most common steps defined in a basic pipeline workflow is the Deploy step. The deployment stage encompasses everything from publishing build artifacts to pushing code into pre-production and production environments. This deployment stage usually involves both development and operations teams logging onto various remote nodes to run commands and/or scripts to deploy code and configuration. While there are a couple of existing ssh plugins for Jenkins, they currently don’t support the functionality such as logging into nodes for pipelines. Thus, there was a need for a plugin that supports these steps.
Introducing SSH Steps
Recently, our team consisting of Gabe Henkes, Wuchen Wang and myself started working on a project to automate deployments through Jenkins pipelines to help facilitate running commands on over one thousand nodes. We looked at several options including existing plugins, internal shared Jenkins libraries, and others. In the end, we felt it was best to create and open source a plugin to fill this gap so that it can be used across Cerner and beyond.
The initial version of this new plugin SSH Steps supports the following:
sshCommand: Executes the given command on a remote node.
sshScript: Executes the given shell script on a remote node.
sshGet: Gets a file/directory from the remote node to current workspace.
sshPut: Puts a file/directory from the current workspace to remote node.
sshRemove: Removes a file/directory from the remote node.
Below is a simple demonstration on how to use above steps. More documentation can be found on GitHub.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Configuring via YAML
At Cerner, we always strive to have simple configuration files for CI/CD pipelines whenever possible. With that in mind, my team built a wrapper on top of these steps from this plugin. After some design and analysis, we came up with the following YAML structure to run commands across various remote groups:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
The above example runs commands from
c_group_1 on remote nodes within
r_group_1 in parallel before it moves on to the next group using
sshUserAcct (from the Jenkins Credentials store) to logon to nodes.
Shared Pipeline Library
We have created a shared pipeline library that contains a
sshDeploy step to support the above mentioned YAML syntax. Below is the code snippet for the sshDeploy step from the library. The full version can be found here on Github.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
By using the step (as described in the snippet above) from this shared pipeline library, a Jenkinsfile can be reduced to:
1 2 3 4 5 6
An example execution of the above pipeline code in Blue Ocean looks like this:
Steps from the SSH Steps Plugin are deliberately generic enough that they can be used for various other use-cases as well, not just for deploying code. Using SSH Steps has significantly reduced the time we spend on deployments and has given us the possibility of easily scaling our deployment workflows to various environments.
Help us make this plugin better by contributing. Whether it is adding or suggesting a new feature, bug fixes, or simply improving documentation, contributions are always welcome.