In this tutorial we’re going to set up a continuous integration infrastructure for a Ruby on Rails application. For this tutorial, I’ll be working from an Ubuntu machine, and we’ll be deploying our code to an Ec2 Ubuntu instance.
Requirement:
Github – source code management tool
Ruby on Rails (rails 5 at the moment)
Any Cloud platform (I’ll be using AWS Ec2)
For this tutorial, we will be cloning one of my repository. The reason why I’m cloning this repository is because it has some tests written in the spec/ folder and we’re testing with rspec-rails (which is already in the gemfile of the application). The test here is focused on the association between the user model and message model, and validation of the message. (After this tutorial, you can try the steps with a new rails application).
LET’S GET STARTED
Clone repository =>
git clone https://github.com/ohiodn8/invoke.git
rename the invoke folder =>
mv invoke cedar
I renamed it to “cedar”
cd cedar
bundle install
(Note: 1. Make sure bundle install passes before you continue. 2. current db for app is sqlite3)
rails db:migrate
cedar is a simple rails messaging application. Test your application on the terminal
rspec
It should pass with 0 failures. (You can start the rails server to try out the application).
GETTING READY TO DEPLOY TO GITHUB
ls -a
rm -rf .git
(remove .git folder)
git init
(to reinitialize repo)
Head over to your Github account and set up the repository.
Click on the plus sign to the right side of the navbar, and slect “new repository”. I’ll name mine cedar, and I’ll go ahead to hit the create button at the bottom of the page.
back to your terminal =>
git add .
git commit -am '1st-commit'
git remote add origin https://github.com/ohiodn8/cedar.git
(change the git url above to yours)
git push -u origin master
Refresh your Github page (on the browser), and you should see your app files on there.
SETUP A CLOUD SERVER (EC2 INSTANCE)
You can use any server you like, but I’ll be using an Ec2 instance/server on AWS (Ubuntu). You can follow this tutorial to help you out.
[NB => While following the Simple Tutorial on how to setup ec2, you’ll get to step 6: Configure Security Group. Add new rule when creating the new security group.New Rule => Open http on port 8080; Custom TCP on port 80 and port 3000 from anywhere[0.0.0.0/0 (Use image for reference) ].
If you followed the tutorial, your Ec2 Ubuntu Server should be ready. You may not be able to connect to the server via the PuTTY if you don’t have it installed on your Ubuntu machine (Yes! You can install PuTTY on Ubuntu.).
Let’s connect via the terminal:
This step is to connect when you don’t have PuTTY. If you have PuTTY or know how to access your server, please skip this step.
[On your instance page on the browser, click the connect button above your running instance you’ll see steps to connect using the terminal]
Start a new terminal on your Ubuntu machine =>
Locate your private key file (e.g tutorial.pem), and from that directory =>
chmod 400 tutorial.pem
Connect to your instance using its Public DNS or IPV4 (e.g.: ec2-23-270-45-109.us-west-2.compute.amazonaws.com OR 23.270.45.109 )
ssh -i "tutorial.pem" ubuntu@ec2-23-270-45-109.us-west-2.compute.amazonaws.com
ssh -i "tutorial.pem" ubuntu@23.270.45.109
INSTALL REQUIRED DEPENDENCIES
At this point, you should have your application repository on Github and your Ec2 instance running. From the ec2 (Ubuntu) server terminal =>
sudo apt-get update && sudo apt-get -y upgrade
Time to install the dependencies on the Ec2 server
We’ll be using ansible to install the dependencies. Ansible is not part of the requirement for this tutorial because no prior use or experience of ansible is required; I’ve written an ansible playbook to install the dependencies we’ll need for a rails app to work on the server. The playbook will install RVM, Ruby, nginx, NodeJS and the gems required for your
rails app to work. The playbook also automates the jenkins installation, but has been disabled, so we’ll have to enable it.
I’ll guide you through the process to install and use the Ansible playbook. Head over to the playbook repository on Github =>
https://github.com/ohiodn8/ansible-rails
(and follow the instruction)
On the terminal serving your ec2 ubuntu instance, run the command below to install ansible =>
sudo apt-get install software-properties-common && sudo apt-add-repository ppa:ansible/ansible && sudo apt-get update && sudo apt-get install ansible
Clone the ansible playbook =>
git clone https://github.com/ohiodn8/ansible-rails.git
cd into ansible-rails =>
cd ansible-rails
nano playbook.yml
(delete # symbol from behind jenkins )
Save the playbook.yml with CTRL X. It’ll prompt you for a yes or no, hit the “y” key and then enter.
Time to run the playbook =>
ansible-playbook playbook.yml
(It’ll take a while for the installation to be complete, please be patient. . .)
When the installation is done, go through the installation process output, and you’ll see some outputs messages.
* 1st output says =>
“THINGS LEFT TO DO:”. Follow the steps below…
ruby -v && rails -v
source ~/.rvm/scripts/rvm
ruby -v && rails -v
* 2nd output returned your jenkins admin password: (copy that and save it somewhere)
432c89bcgdf64159ce4ae0dacbda87d [this is an example of how the jenkins password should look like]
At this point, we have our ec2 instance ready with the required dependencies, and jenkins running on it. On your browser, visit the IPV4 address at port 8080.
Example (use your own address) => http://23.270.45.109:8080
[NB: If the link didn’t work, it maybe because you didn’t open port 8080 on the Security Group of your AWS EC2 Instance. Find Security Group and open port 8080]
If the link worked, You should see a bold “Unlock Jenkins” page, and a form asking for an administrator password, copy the password you copied from the ansible installation.
Select the “Install Suggested Plugins”, and when that’s done, continue to Create First Admin User.
Complete the form and hit the Save and Continue button at the bottom of the page.
You’ll see a Jenkins URL: http://23.270.45.109:8080/ [That’s for me]. Hit continue and …. (complete the rest of the process).
ACTIVATING THE WEBHOOK ON GITHUB TO CONNECT TO JENKINS
Now let’s head over to our application repository on GitHub. On your Github page, hit the Settings tab [like you see in the image below]
Then click the webhooks tab. On the Webhook page, click the Add Webhook button and paste your jenkins webhook ( your jenkins url + /github-webhook/ ) Example below =>
http://23.270.45.109:8080/github-webhook/
. . . in the Payload URL section… and save.
Head back to Jenkins on your browser to configure Jenkins to work with your application
We have our project ready and hosted on Github and Jenkins installed, all we need now is to integrate both of them so that when we push any change to Github, Jenkins know about it and do the testing and deployment.
Click on “New Item” and fill in the proper project name. Choose “Freestyle project” among the options, and move to the bottom of the page to click okay.
In configuration screen, check Github project and add the Github project link.
Under “Source Code Management” select Git and add the Repository URL with .git extension.
Under “Build” section select “GitHub hook trigger for GITScm polling” option.
Under “Build” section, Click on “Add build step” button and then click “Execute shell” option.
In the execute shell =>
pwd
The command here is not important; this will just check for the path where are rails app is saved on the jenkins user. . . We’ll handle the command later with a bash script.
[SAVE]
Head back to your rails app (on your local machine) and make a change to your application. You can make use of the editor you prefer, I’ll use nano via the terminal
Run =>
pwd
(you should be in your cedar folder)
nano app/views/messages/index.html.erb
Change any static value. For me, I’ll change . . .
=>
Save the page. From the root of your application (still in the terminal on your local machine) =>
git commit -am 'little-changes'
git push -u origin master
Now head over to your jenkins page (on browser) and the process should pass with a blue tag by the side.
Hit the blue tag or the console output link, and you should see “SUCCESS” at the bottom of the page. (if it doesn’t pass, you should get a red tag)
And the pwd command in the build shell returned “/var/lib/jenkins/workspace/cedar”
BASH SCRIPT
Time to write our bash script. Our bash script will be handling the rspec t.
Quick Scenario => from this point you can run a bundle && rspec from the build shell, but for this to work, you’ll have to give the jenkins user sudo rights. You can then add a bash script to update changes to another server if rspec passes. You’ll also want to add capistrano to the app to handle final push to production and setup another server for production. If there’s any fail, you can setup jenkins mailing to send out emails. We won’t be doing this scenario as its not within the scope of the tutorial. . . (really not in the mood to spin up another server and enable sudo rights).
Our Scenario => for the remainder of this tutorial, we will write a bash script to allow the jenkins user to ssh into our current user on the same server, and clone the github repository , and then upload changes to the browser so we can view it.
DEPLOYMENT
On your ec2 server, let’s login to Jenkins user and switch to root user =>
sudo su
(Jenkins automatically creates a new user after installation. Switch to it using this command =>)
su - jenkins
Let’s generate RSA key, run following command =>
ssh-keygen -t rsa
(if it prompts you for anything, hit the enter button)
Your public key has been saved in /var/lib/jenkins/.ssh/id_rsa.pub
cat ~/.ssh/id_rsa.pub
Now, copy the pub key (and save it somewhere). Head back to ubuntu user. . .
exit
exit
(. . .should take you back to the ubuntu user)
Lets clone our repository here and make sure it works. . .
git clone https://github.com/ohiodn8/cedar.git
(do not copy this, copy your own repo)
cd cedar
bundle install
rails db:migrate
rails s -d -b 0.0.0.0
Your app server should be running in background. Check to see if its working by visiting the IPV4 address at port 3000
e.g: 23.270.45.109:3000
(you should be greeted by an authentication page on your browser )
Great! Now, you have a running server in the background.
Back to your ec2 server terminal
cd ..
pwd
[ you should be here /home/ubuntu]
switch to ~/.ssh directory using following command ==>
cd ~/.ssh
nano authorized_keys
Put the pub key in there (in addition to the one you find)
In order to validate whether keys are properly configured or not, switch to Jenkins Server and try to login to ubuntu user using SSH.
e.g.
sudo su
su - jenkins
ssh userName@ubuntu-IPV6-ip
For me => ssh ubuntu@171.54.37.206
(You exit by typing exit)
LET’S WRITE THE DEPLOYMENT SCRIPT AND FINISH THIS JOB
In the local machine terminal, create a new file from the application root directory =>
touch deploy
nano deploy
Inside deploy =>
#!/bin/shssh ubuntu@171.54.37.206 <<EOF cd /home/ubuntu cd cedar git pull https://github.com/ohiodn8/cedar.git bundle install && rails db:migrate && rspec specEOF(the info here will not work for you. Update with your own info)
What just happened =>In here you told jenkins to ssh to the ubuntu user at the IPV6 address, find its way to the root directory of your application, and make a git pull, and the rest is understandable. (double check to make sure you have the right info. Use your IPV6 in stead of the mine on the script, and change the git repo address to yours).
Make the file executable using the following command. This is important for Shell script to execute =>
chmod 750 deploy
chmod +x deploy
Go to Jenkins project page, from left navigation click on “Configure” and scroll down to “Build” section.
Add “./deploy” in the build commands.
Now from your local machine terminal, push the app to Github.
git add .git commit -am '2nd-commit'
git push origin masterHead over to Jenkins on the browser. . . and it should pass.
[Well for me during the time of this tutorial, it failed with a Connection Time Out, so its a red tag. This is because I didn’t change the IPV6 from the fake in this tutorial. I copied my IPV6 from my ec2 web page and re-pushed the app from my local machine]
If you have any fail, check the console output and debug the issue.
Refresh the app url on the browser (example: http://23.270.45.109:3000).
You should still be on the authentication, create a new user account and your updates should appear on the message page. You can make changes from your local machine and deploy to Github, and your changes should appear on your application.
Let’s make a change that would cause rspec test to fail . . . for example, in your message.rb model file, take out the belongs_to :user line and push your app to Github. Your Jenkins build will fail. Fix the issue and re-push to Github . . .
You may be wondering why I have rails db:migrate on the bash script; for example, if you create a migration on your local machine and make a push, the bash script will run the migration and upload the changes to your cloud server. Let’s try it.
Local Machine terminal =>
rails g scaffold Post title:string
rails db:migrate
git add .
git commit -am 'post-added'
git push origin master
Your jenkins webhook should catch the build, and it should pass. Now, head over to your browser and visit Your-IPV4:3000/posts
(example: http://23.270.45.109:3000)
And that’s all. There’s still alot to be done to improve the environment but we’ll stop here. You could also try the quick scenario I described in this tutorial.
[Remember to terminate your running instance. . .]