Getting Started with the EC2 Plugin For Jenkins

For this How To, I’m going to assume a few things:

• You have a Jenkins server configured in AWS
• You have the EC2 Plugin Installed (its called Amazon EC2 Plugin)

EC2 Policy For Jenkins

First thing we’re going to do is create a Policy for our Jenkins server. This Policy will allow jenkins to talk to EC2.

If you have terraform configured then great, feel free to build a Policy using that. For the purpose of this guide we’ll be doing everything step by step.

1. Login to AWS Console (https://console.aws.amazon.com)
2. From the Services drop down on the top bar, select IAM
   

   

3. Select Policies on the left hand side and then press “Create Policy”
   

   

4. On the folders tab select JSON
   

   

5. Copy the policy below into the json text editor. This policy is the standard policy that the plugin developers suggest.
   
6. On the Review Screen, Name your Policy – Make note of this you’ll need it for the next step.
7. Provide a good Description. I like to use something like:
   “This Policy is used for jenkins to start and stop slaves on “servername”
8. Then Press Create Policy

EC2 Role For Jenkins

I prefer to use Roles vs Users and API Keys for a few reasons:

• Roles only work on the instance they’re defined
• There is no API key that can be used elsewhere
• These keys automatically rotate

1. Select Roles on the left hand side and press Create Role
2. On the Role type screen press AWS Service, select EC2. Then, press Next: Permissions
3. Find the Policy you previously created, select it and press Next: Tags
4. You may skip tags if you want, by Pressing Next: Review
5. On Review Screen, Name your Role – Make note of this you’ll need it for the next step.
6. Provide a good description I like to use something like:
   “This Role is used for jenkins to start and stop ec2 slaves on “servername”
7. Press Create Role

You’ve now created the Policy and Role required for Jenkins to start slaves.

Attach your EC2 Role to your Jenkins Master

1. Logon to the AWS console
2. From the Services drop down on the top bar, select EC2
3. Select Instances from the left navigation Menu
4. Find and Select your Jenkins Server
5. Press Actions then from the drop down select Instance Settings –>Add / Replace IAM Role
6. On the Add/Replace Role screen, find and select your role, then on the right hand side click Apply.

Done! Jenkins now has a IAM role attached to the EC2 instance that can be utilized to Create Slaves.

Create a security group for jenkins to talk to it’s slaves

1. Logon to the AWS console
2. From the Services drop down on the top bar, select VPC
3. On the left hand side  under Network and Security, select Security Groups
4. From the top press Create Security Group
   
5. In the security group window input as needed, in the picture below, we’re using the security group for Jenkins to allow all inbound traffic to the security group of our slaves.
   
6. Press Create
7. Make note of the security group name you will need it shortly.

Find the subnet you will be using

1. Logon to the AWS console
2. From the Services drop down on the top bar, select VPC
3. On the left hand side under Virtual Private cloud, select subnets
4. Find the subnet ID you want your jenkins slave to run in. Make note of this, and the region’s availability zone
   

In our case our subnet would be subnet-14740d7c in us-west-1c

Find the AMI ID

1. Logon to the AWS console
2. From the services dropdown of AWS, select EC2
3. On the left hand side Under Instances select Instances
   
4. From the top bar select Launch Instance
5. On the next page, find the AMI and OS you’d like to use. For this how to we’ll be configuring Linux.  I prefer ubuntu so my ami would be ami-06397100adf427136

Create a SSH Key in AWS

1. Logon to the aws console
2. From the services dropdown of AWS, select EC2
3. On the left hand side Under Network and Security select Key Pairs
4. Find and select Create Key Pair
5. Name your Key Pair. Since you may have different regions, and services i’d suggest something like servername-jenkins-slave-<region>
6. Then select Create
7. AWS will present you a file to download, it will only allow you to download this once. You’ll need it in the next step.

Configure EC2 Plugin

1. Logon to your Jenkins server
2. From the left hand side select Manage Jenkins
   
3. Now select Configure System
   
4. Scroll to the very bottom of configure system until you find “Add new cloud”
   
5. Press the dropdown Add a new cloud and select Amazon EC2
   
6. Name your integration, I usually use AWS Linux Oregon (or other religions). 
7. Select the checkbox Use EC2 Instance Profile obtain credentials
8. Select your Region
9. From the key pair we generated above, open the file in your text editor and copy its contents into the EC2’S Key Pair Key box.
10. Click Test Connection you should receive a success

Add AMI to EC2 Plugin

1. Under the AWS configuration in Jenkins you just created find and select add AMI
   
2. Now we’re going to fill out the ami creation with information we’ve gathered above.
   1. Description – This can be anything you want, it shows up on every node name in jenkins.
   2. AMI ID – This is the ami you want jenkins to start, as referenced above we’re using ami-06397100adf427136
   3. Instance Type – Use your best judgement (I prefer T3 Series currently)
   4. Availability Zone – this corresponds with where your subnet is: For our example its us-west-1c
   5. Use Spot Instance – Since these nodes are usually designed to be spun up and torn down, it makes sense to run them on the spot market.. But again use your best judgement
   6. Security Group Names – This is our security group name, as referenced above we’re using jenkins-slave
   7. FS Root – This is where your jenkins workspaces will be ran from.
      • For most cases the user home directory will be fine
        • For Ubuntu its /home/ubuntu/
        • For Centos it’s /home/centos/
        • For Windows it could be a folder you’ve made C:\jenkins
   8. Remote User – The user you’d ssh or winRM into your slave
      • This varies by OS
        • For Ubuntu it’s ubuntu
        • For Centos its centos
        • For Windows you’d use administrator
          • Windows will also require a static password, at time of writing dynamic is not supported
   9. AMI Type – Either unix or windows 
   10. Labels – This will be the label you’ll use in your job to call a slave
   11. Usage – We usually change this to “Only Builds with jobs  label expressions matching this node”
   12. Idle Termination time – For purpose of testing I set this to -5
   13. Init Script – Since there’s no source control here, i’ll usually pull my init script from a source code repository. (you’ll need to use sudo to execute)
   14. Number of Executors – Set this to a reasonable level, usually 2 by default
   15. Subnet IDs for VPC – The subnet identifier you want to start this slave in,  as referenced above we’re using subnet-14740d7c
   16. Maximum Total uses – This lets you select how many times jenkins can use this node before it has to destroy it. By default its -1 (unlimited)
   17. You’re done, you can configure other options as required
   18. Find and select the save button

Testing our slave startup

1. On the left hand side of jenkins find and select Build Executor Status
2. On the drop down – Provision via Linux AWS Oregon (or your aws connection name) Select your newly configured AMI
3. Jenkins should now start provisioning your new slave. It may also indicate where you have a misconfiguration
   • You can also look at the executor logs itself on the slave to see where further problems exist