Over the past year I’ve become more and more familiar with the terminal and (a limited set of) the commands that are possible. Swapping out GitHub Desktop app for the Git terminal commands and learning about moving around, manipulating files, VIM (read: I use Nano) and other possibilities in this new world, I was now ready to tackle something new: writing a shell script.
What originally drew my attention to doing some shell scripting was hearing that Adam Wathan of Full Stack Radio originally wrote Nitpick CI, his PSR-2 coding standard app, in PHP but later converted it to shell scripts. I thought: “Dang! I should check out some shell scripts and learn when they come in handy”.
After putting in a bit of Googling and Stack Overflowing, I’ve hit the nail on the head and got my simple script running. Pretty stoked!
Zero downtime deployment
Something that has been bugging me recently is my website deployment setup for myself and my clients. I want to be able to run a single command to retrieve the new site from the Git repository and then run any composer commands or additional shell commands necessary (e.g. compiling assets with Laravel Mix).
The problem with running these commands after deploying from a repository is that it all takes time, and it means that for however long all these commands take to run, the site is in an incomplete state for any requests that come through.
Enter: zero downtime deployments
The idea here is that you can run all these commands and while they are running the previous site is still active. Once the commands have finished running and the new version of the site is ready for visitors, the site delivered to the user is switched out seamlessly with the new one.
There are plenty of tools out there to do this for you. Check out Envoyer. But what developer ever thought: “Hey it exists so I don’t have to build it myself” …right?!?
Yes…I named it 😎 …and created a Github repository for it.
Creating a deployment directory
The first step is to create a directory in which the fresh git repository will be cloned into. I used to just use
git pull command, but this meant that there would be some downtime.
From the server root we will create the directory structure
flipit/deploys/UNIX_TIME. Unix time is simply the seconds since midnight 1970. This structure will mean that each time we create a new deployment directory this number will be different and we will be able to determine the order of the directories easily.
To create this directory structure, we need to run the following shell commands:
If at any point you want to trouble shoot and check the value of the variables you create, you can output their value by using the
echo command, e.g.
echo $DEPLOYMENT_DIRECTORY. Also, if any of these commands are confusing, or you would like to know more about them, you can bring up the manual by typing
man COMMAND_HERE, e.g. to learn more about the
mkdir command type
Pro tip: if you’re all 😱 and cannot get out of the manual, type
q to exit the manual! Took me a few tries to work that one out 👍
So now we have our directory ready to roll. We are killing it! 🤘
Cloning our repository to our deployment directory
If you’ve been working with Git for a while, this step is what you’ve probably done a thousand times before. We are going to get our system to clone our repository from GitHub into our deployment directory, instead of using
git pull in the live directory, which is usually live…and would cause downtime…that is the thing we are trying to avoid…you get it.
Note: You will need to have setup SSH keys for this to work seamlessly. GitHub have a help section to guide you through the setup process. When I was new to all this, I didn’t really get SSH keys, but having spent a bit of time working it all out, it is a great help and makes things so much easier.
To make this magic happen, we will first need to get the repository URL from GitHub:
and then run the following shell commands:
Once this has run, we now have our fresh repository waiting for us in our deployment directory. Navigate into the folder to check it out. It is finally time to FlipIt™!
Switch out the current site, instantly!
This step is basically just creating a shortcut, just like you would on your computer when you might create a shortcut on your desktop to an application you use, or a file you are constantly editing. We are going to tell the system to create a shortcut from our public directory to our deployment directory. Switching this shortcut is what gives us the zero downtime.
In my repository I have a
public_html directory which has my current site ready for deployment. If this was a Laravel installation the repositories public directory would be
Now our servers
public_html directory is actually a shortcut to our repositories
✨ MAGICAL 🦄
Let’s wrap that all up together now.
Converting commands to a script we can run
You don’t want to have to remember all this and type it every time, plus have to change the variables for every server you own, so instead, let’s make this into a script you can call and run from your local machine to FlipIt™ on your remote server, providing the directories and URL as arguments.
Making and running a script file
To create a script file we could use a text-editor, but while we are doing commands, let’s use the terminal to do it. First we need to create a file, but we will also need to allow the file to be executed.
Now you can open the
flipit.sh file in your text-editor of choice and fill it out as we go.
Shell scripts allow you to provide flags like
-p, as you’ve seen above, but also arguments. As a example say we want to call a script that logs you into a system, we would need to provide a username and password. This is done via arguments and might look something like this…
As we are going to provide our URL and directories as arguments when we call the script, we need to add a little bit of extra code to our script, that I found online. Being new to scripting and the syntax, I get what this new block is doing but it’s a copy and paste job…it loops through the provided arguments and allows you to use the values provided by the user. In our case, we are just going to set our
REPOSITORY_PUBLIC_DIRECTORY_PATH variables in this loop.
So let’s move our variable creation into our loop that detects the provided arguments. Because we are going to save this script as a file, we will also need to tell the system what type of script it is by adding
#!/bin/sh at the top of the script file so that the system knows how to interpret the commands, just like our HTML
<!doctype html> at the beginning of a HTML file tell the browser how to interpret the HTML it’s about to parse.
Once this is in place, these values can be provided by calling:
Calling the script locally, but executing on a remote server
The last piece of my puzzle was to be able call the script on my local development machine, but have it login to the remote web server and execute the script on a remote server. Because I have my development and web server matched up with SSH keys, I can easily SSH in and run these commands, but now I want the script to SSH into my server and execute the script automatically.
Just to reiterate, learning about SSH’ing into servers has been a very helpful skill. I’ve added my server to my
.ssh/config file so I can call
ssh timacdonald to SSH into my web server without having to provide the login details each time.
So now I just need to be able to get my script to SSH in and run the script. Luckily for us, we can pass the script as a string and pass it as an argument to the
ssh command. To do this we’ll wrap the script to run in a heredoc. Let’s see what that would look like:
Of course, you will probably want to make the host an argument the user can provide also so that it is not hard coded into the script.
And that’s a wrap on my goal - Nailed It! This is really just an exercise for me to wrap my head around shell scripting and the basics of how it comes together, but I’m really happy with the outcome. Now I can run the script and have my site updated to the latest commit with zero downtime, and if you’re reading this post - it WORKED 💪
- I’d like to add a rewind option to flip back to the previous deploy in case something goes wrong.
- Having an
initoption for FlipIt™ that stores the repository and folder variables in a configuration file so that after the initial
initcommand is called, you no longer need to provide the folder and URL arguments.
- For my purpose this is zero downtime, however I’m not 100% it’s an atomic swap. You should really just use Envoyer for any serious production stuff, this is just a toy.
- This is working for my setup, but different servers might have different setups and the code may not execute as expected. I’m running these commands on my development Mac. I’m not expecting this to ‘just work’ for you, but hopefully in following along you did learn some things, I know I did!
- The script does not currently clean up after itself i.e. after many deploys you may start to fill up your web server. For now you will want to manually clear out old deploys from the deploy directory. In the future I might add a cleanup script to remove the older deploys…or you could just use Envoyer.
- I’ve just worked this out through trial and error. Dig into shell scripting, it’s a different world and good fun experimenting with - makes you feel like a programming guru at the cafe as well 🤓.
- If you are running something like Laravel and wish to run any additional command, such as
npmcommands, you can slot them in before the symbolic link is made, but make sure it’s not AFTER the link is made, otherwise our Zero Downtime is gone e.g.
Well you made it this far, so it you’d like to discuss anything about the article, or criticize my lack of knowledge, please feel free to hit me up on Twitter.
This will ensure that only the latest version of the files is downloaded, instead of including every revision of every file in the repositories history. Thanks Matt!