A while ago one of my production systems went down quietly. No crash logs, no alert, nothing dramatic — a cronjob just stopped firing and an endpoint started failing checks that nobody was running. I only found out when a customer emailed asking why things looked broken. By then it had been down for hours. That’s the moment “it’s probably fine” stops being an acceptable monitoring strategy, and the moment I decided to build the tool I actually wanted instead of settling for a half-fit off-the-shelf dashboard.

That tool is Checkmeup, and it’s now running in production, watching my own projects. This post is both the origin story and a full walkthrough of what it does.

Once again, I come across the situation where our infrastructure becomes a mess, and we migrate everything to Terraform, and in the latest iteration, to AWS CDK. But then the real mess begins. Eventually, everyone is afraid to make any changes, fearing that something might break, and as a result, nothing gets changed. The snowball keeps growing and becomes increasingly unmanageable. At some point, I got tired of it all and decided that we need to bring order, but in the least painful way for the developers.

We decided to go back to Ansible for automating small tasks needed to manage our infrastructure. In this case, the fear of causing harm was much less, and the code became more readable and understandable. Overall, I enjoyed it. So I decided to share my experience with you in creating such small playbooks for managing our infrastructure. In this article, we will create a VPC using Ansible.