Automatic rsync deployment with Github Workflows

Updated on July 29, 2023
Updated the fact that this blog is now built with Hugo instead of Jekyll

As an indie developer, I used to manually deploy my web applications to my VPS. But the more and more subsystems and apps I tend to build, the more my server get crowded with new applications to deploy and maintain.

As I write those lines, my VPS powers:

• This blog (Hugo)
• Padlok API (Symfony)
• Padlok Share WebApp (Vue.JS)
• Padlok Marketing website (Hugo)
• Pet feeder API (Symfony)
• A friend audio stories (Jekyll)
• … and more!

By multiplying the number of services, this process became time-consuming, but it was also prone to human error. It was time for me to automate most of those processes

Motivations

Aside the logical time-saving and reliable deployment, what really triggered my shifting to this solution was storage consideration.

Not that I specially want to become a storage Scrooge, but my VPS is only 20G; and I’d prefer to increase this storage only because of business reason, and not because of my manual deployment method.

Manual deployment rely on Git. Git comes with a special folder .git that ends up to contains all the history of the repository. Not relying on git anymore ends-up to save a lot of space, as long as we do not deploy this folder anymore.

Also, the static web generator tools like Jekyll or Hugo are software that will generate the entire website to serve in a subfolder (_site for Jekyll, and public for Hugo). The same goes for the Vue.JS webapp, that will generate a standalone dist folder.

Aside those generated folders, all the “source code” of those websites are also not necessary on the server. It prevents to have assets like images/videos in double, and also save some more space.

Finally, plenty of tools that were installed on the server (Ruby, Hugo, Jekyll and all the dependencies) were installed on the server to perform the generation, and will no longer be of use. Again some space saved on the disk.

As a result, I saved 30% of my total storage by moving to automatic deployment.

Github workflows

GitHub workflows are a powerful tool for developers to automate their workflows and streamline their development process.

A GitHub workflow is a set of automated actions that are triggered based on specific events, such as a push to a repository, a pull request being opened, or a new tag being created.

Workflows are defined in yaml files, and located in the .github/workflows directory right in your repository. It’ll include:

Name

The name of the workflow that will be displayed in the GitHub Actions tab.

On

The event that triggers the workflow, such as a push to a specific branch, a pull request being opened, or a new release being created.

Jobs

The set of actions that should be performed when the workflow is triggered. Each job can include a set of steps, which define the individual actions that should be performed, such as building the code, running tests, or deploying the application.

Environment variables

Optional variables that can be defined for use in the workflow.

Notifications

Optional notifications that can be sent to team members or external services, such as Slack or email, to alert them of specific events in the workflow.

Deployment process

To build and deploy the apps, there are few steps, some optional, to ensure stability and consistency of deployments.

1. Checkout the code : A must-have. We do need the code to build things
   → actions/checkout

2. Install the required tools : Depending on the app we build: Ruby, Hugo, Node, PHP, Composer…
   → ruby/setup-ruby
   → peaceiris/actions-hugo
   → actions/setup-node
   → shivammathur/setup-php

3. Install the code dependencies : Using the dependency manager, like Composer, Bundler or npm.

4. Run the test suites when applicable : To make sure we won’t break something, write tests, lot of tests. And run them. Always.

5. Remove the dev dependencies when applicable : We do not want to deploy some useless dependencies that are only used to run the test suite, when dependencies are shipped with the code (like with composer based app).

6. Rsync only the result folder : Rsync is powerful because it’ll only send files that have been changed, preserving i/o processing and bandwidth. Of course, we will skip useless folders and files like .git.
   → burnett01/rsync-deployments

7. Run post-deployment scripts when applicable : Some apps need to run database migrations. Others will require an associated daemon process to be restarted.

Dealing with secrets

Rsync-ing requires secrets, like SSH port, SSH private key. Those should never be committed.

Instead, Github provide a secret handling mechanism directly in the repository settings:
Go to your project > Settings > Secrets and variables > Actions

From there, add as many secrets as you may require, like host, port, user, ssh key and deployment folders.

Examples with my workflows

Hugo deployment

→ Dean’s blog

Jekyll deployment

→ Affres de Reivax

Symfony deployment

→ Padlok API
→ Pet feeder API

Vue.JS deployment

→ Padlok Share webapp

Closing remarks

Discovering, and using Github workflows; as well as reconfiguring my server to run the newly deployed code, aside with removing the old now useless tools from it took me quite some time. But I’m sure that it’ll work out in the long run.

Haven’t calculated how much, but it reminds me the good old XKCD’s about task automation.