Sidekiq with environment variables and systemd

Normal setup

Running Sidekiq with systemd is easy, download the sidekiq.service file from Github and place it in /etc/systemd/system.

Start it with: sudo systemctl start sidekiq

Enable it to run at boot: sudo systemctl enable sidekiq

And stop it: sudo systemctl stop sidekiq

The only lines you have to change are:





With environment variables

But what if you have some environment variables that you want to load before starting Sidekiq? Keep in mind that there are many ways to do this, and this only how I do it.

I put the environment variables in a hidden file called ‘.env’ in the home directory of the user that runs the application. If this user is called deploy, the file would be /home/deploy/.env

In it are my environment variables with export in front of them:

export DATABASE_URL=postgres://deploy:sdfsdfDNFSDF@localhost/my_prod_db
export SECRET_KEY_BASE=640c916asdfasdfasdfasdfasdfasdfasdfasdf
export SMTP_ADDRESS=""

These are loaded when you log in through ssh with source as the first line in .bashrc. It doesn’t have to be the first line but it has to be above the “if not running interactively don’t do anything” line.

source ~/.env

But how do these env variables reach Sidekiq? We’ll simply start Sidekiq with bash in the Systemd service file. Change the ExecStart line in /etc/systemd/system/sidekiq.service and add /bin/bash -lc in front.

ExecStart=/bin/bash -lc '/usr/local/bin/bundle exec sidekiq -e production -C /home/sysadmin/checkout/config/sidekiq.yml'

Bash will load the environment variables and then start Sidekiq.

Ruby on Rails on Windows 10 in 2019

If you want to install Ruby on Rails on Windows, look no further! I recommend that you use either Linux or macOS to develop in Rails if you have the option. But sometimes you have no choice. My Macbook has to go back to Apple for repairs so I decided to try developing on Windows. Here is my setup:

Install WSL

You can use Ruby in Windows with Ruby for Windows but if you want to use Rails I recommend that you use WSL. WSL is short for Windows Subsystem for Linux and it creates a Linux environment in Windows. WSL is fully supported by Microsoft so you can get it for free from the Microsoft Store:


But don’t start it after the install finishes. Open a Windows Powershell with Administrator rights.

And copy and paste this line:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

Hit “Enter” and restart your computer.

If you don’t enter the “Enable-WindowsOptionalFeature” line you will get the following error and WSL won’t work:

Install updates and dependencies

Now that you have WSL installed it’s time to start it. The first time it starts you are asked to create a user. Go ahead and create one, make sure you remember the password!

Before we start installing stuff we have to update the Linux packages. Copy and paste this line to update everything:

sudo apt update && sudo apt dist-upgrade && sudo apt-get autoremove && sudo apt-get clean

Tip: you can right-click to paste something in the WSL terminal.

It’s a good practice to run this line once a month to keep things updated and safe.

Install Ruby and Rails and dependencies

Just a tip: if you hold shift while clicking on the Ubuntu icon you open a second window.

Rails has a lot of dependencies, let’s install them:

sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev software-properties-common libffi-dev libreadline-dev zlib1g-dev

Before we can install Rails we need Ruby first. There are many ways to install Ruby:

  • from apt with sudo apt install ruby2.5
  • with RVM
  • with rbenv
  • compile from source (hard mode)

Let’s go with rbenv because that is the most common way:

curl -fsSL | bash

and add rbenv to the PATH variable, like it asks for after installation:

echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL

Be careful when copy & pasting, sometimes Windows turns single quotes (‘) into backticks (`) or other curly quotes.

When rbenv is installed you can use it to install Ruby and activate it. No need for sudo this time.

rbenv install 2.5.3
rbenv local 2.5.3

TIP: I’ve noticed that WSL sometimes seems to hang, usually selecting the window and pressing a random key is enough to activate it again. So when the commands above seem to take a very long time press the spacebar every now and then to make sure it’s still working.

Time to install bundler and then Rails:

gem install bundler
gem install rails

You now have an up to date WSL environment with rbenv, ruby 2.5.3 and the latest bundler and rails installed.


I’ll do a writeup of installing Postgresql in another post. But it basically comes down to two choices:

  1. install Postgres inside WSL
  2. install Postgres with the Windows installer

My recommendation is to use the Windows installer because I ran into some weird problems when trying to use Postgres in WSL. Use the interactive installer, not the Graphical installer by BigSQL.

Some tips and tricks

Ping me on Twitter or email if you need more help. Or look at one of the guides other people have written:

GoRails: Rails on Windows 10



Clients website going offline at random

Have you ever been asked for one of those cases that nobody can figure out? Or do you have one of those recurring problems that are impossible to troubleshoot? Maybe you fix them with a reboot when they occur. I love cases like that. You can’t always fix them, but you always learn something. I had one yesterday and I managed to fix it. Here are some approaches that I used to solve it.

What was the setup?

This client had two servers. One with some WordPress sites and one with their application. The users connect to the application server and this server is also the proxy to the WordPress sites. This is an easy way to make it look like the WordPress pages are part of the application. When the user is filling forms they interact with the application and when they click on documentation they get a WordPress page. And because the application server proxying this page, it looks like the documentation is in the application. It’s on the same domain and uses the same layout. Pretty neat.

Making a quick sketch of the situation is often a good idea. It shows the client that you understand their setup and their problem. And it gives you insights that you wouldn’t by just clicking around.

What was the problem?

According to the customer, the WordPress server would be completely unreachable and they had no idea what triggered it. Maybe Digital Oceans network was to blame. They solved it temporarily by changing Nginx configs, connecting the proxy to internal addresses instead of public addresses and flushing firewall rules. Oh yes, and lots of reboots.

Don’t believe the customer

Like any detective show on television, server problems also come with a lot of misinformation. My client was blaming the Digital Ocean network for being unreliable. But Digital Ocean has a status page, so I went there first. According to them, everything was fine when the problems occurred. So I noted that as their alibi.

When I logged into the server it turned out they were using Apache and not Nginx. And when I asked them if the WordPress pages were unreachable for everyone or just the production server, nobody knew.

Test it yourself

I did not get a chance to experience the unreachable state myself. Because the downtime was a big problem for their customers and they would fix it as soon as it happened. If you get the chance to see a problem live that’s always better than relying on reports from eyewitnesses.

I had 3 approaches left to follow:

  1. look around on the server
  2. read the logfiles
  3. read the config files.

The solution is probably in the log files but I choose to look around on the server first. Just a quick check to see the uptime, the load, disk usage, last logins and what is running.

My look around showed nothing that made me suspicious. I used ‘w’ to see uptime, load and logged in users with one command:

user@server:~$ w
 12:51:47 up 2 days,  2:11,  1 user,  load average: 0.25, 0.22, 0.15
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
user     pts/0    213.127.192.xx   12:49    1.00s  0.17s  0.01s w

Read the logs

The log files are the place where you usually find the murder weapon. Or in this case, the line that will explain the problem. I always start with the syslog. It’s in /var/log/syslog. If it’s not too big I’ll just open it with Vim or Nano and scroll through it so I get an idea what is going on. If it is big I’ll grep it for keywords like error, block, killed, etc.

After that, I read individual log files starting with the ones that are the most system and networking related. For example the kernel log, firewall log, fail2ban log. And then the logs from applications that run on the server like the webserver and the database server.


This time it was fail2ban that blocked the server with the WordPress sites because it was sending too much traffic. I disabled fail2ban and the site has been up ever since!


How to enable pruning on bitcoind

If you don’t want to have the whole bitcoin blockchain on your computer or server you don’t have to. There is an option that deletes previously verified blocks from your local copy. This is called pruning. To enable it you have to add the following line to your bitcoind config. This config file is usually in ‘/etc/bitcoin/bitcoin.conf’.


After that you have to restart bitcoind and 5 minutes later you have a whole lot more free diskspace.

You can check if pruning is enabled with:

bitcoin-cli getblockchaininfo | grep prune

It will output something like this:

  "pruned": true,
  "pruneheight": 552518,
  "prune_target_size": 1073741824,

Good to know:

The prune= value has to be higher than 550. And the number indicates the diskspace to be used in Megabytes. I went with 5000 because 5GB seemed reasonable to me.


After I enabled the prune=5000 option bitcoind wouldn’t start anymore. There was no usable error:

root@cryptodoos:~# service bitcoind start
Job for bitcoind.service failed because the control process exited with error code.
See "systemctl status bitcoind.service" and "journalctl -xe" for details.

I checked for typo’s in the config file and read the logfile in /var/lib/bitcoin/debug.log but everything seemed fine. So I started the daemon by hand. This is usually a good approach to see what’s going on.

root@cryptodoos:~# /usr/bin/bitcoind -daemon -datadir=/var/lib/bitcoin -conf=/etc/bitcoin/bitcoin.conf -pid=/run/bitcoind/
Error: Prune mode is incompatible with -txindex.

And there it is. You have to disable txindex in the bitcoin.conf to use pruning.

How to use HMAC-SHA256 to connect to a REST API like Ticketmatic

A client recently asked me to export records from Ticketmatic. Ticketmatic is a SAAS application for selling event tickets. They have a JSON API, so I figured it would be easy. Just send a GET request to some URL and parse the result as JSON right?

That doesn’t work because they use a hashing algorithm called HMAC-SHA256. This requires you to sign every request you make with a secret key to create a signature. After that, you have to put the signature, the current timestamp and an access key in the Authorization header of the request. Not just once but for every request!


Continue reading “How to use HMAC-SHA256 to connect to a REST API like Ticketmatic”

Multiple Passenger apps on your server? Give them names!

If you run multiple versions of a Rails application on the same server it’s easy to get them mixed up. I have a client that has 2 versions of the same application on the same server. One version for clients in the Netherlands and one for Belgium. Because they are almost, but not totally, identical I’m doing as much as I can to make them easy to identify. And today I found a simple trick that I’d like to share.

Continue reading “Multiple Passenger apps on your server? Give them names!”

Is Digital Ocean’s One-click app for Ruby on Rails any good?

Digital Ocean offers 2 types of droplets (servers):

  1. Droplets with a clean Linux install.
  2. Droplets with some application preinstalled: “One-click apps”

Let’s have a look at the One-click Rails installation they offer. I’ll describe what you get and what I like about, what I don’t like about it and I’ll give some tips on how to use it.

Welcome to One-click apps, what do they do? Do they do things? Let’s find out!”

Continue reading “Is Digital Ocean’s One-click app for Ruby on Rails any good?”

Use a git-hook to deploy your app

There are a lot of ways to deploy an app to a server, here is a simple one that I often use. This can be a bit confusing if you are not familiar with Git but I promise it’s the easiest way!

It works like this: you create an empty git repository on the server and you push the branch you want to this reposity from your development machine. The repository on the server has a little script (hook) that puts the files in the right directory (‘rails_project’) and runs all the bundle commands. You end up with 2 directories: the bare repository and the ‘checkout’ with the project. Continue reading “Use a git-hook to deploy your app”