Report: “We went Monorepo.”

This post was originally written by Gunar Gessner. Reposted with permission.


In October we decided to make a horizontal investment in speed and decided to go monorepo with our services architecture. Naturally, we still want to be deploying to microservices for scalability and resilience — so it’s not a monolith, it’s a monorepo. And I was entrusted with the implementation.


The benefits.

There are quite a few benefits to the monorepo architecture, and three of them stood out for us.

1. Code review.

Code changes to multiple different microservices can be contained in a single pull request. So you have the full scope of the change in a glimpse. You can even release new versions of packages and update the services that consume them at the same time.

2. End-to-end testing.

We must admit, in favor of being nimble we’ve been skipping writing E2E tests, but now for DAL2 we’re ready to make the commitment. We want to write Unit Tests, then E2E Tests, and only write Integration Tests where absolutely necessary (because in my experience, the latter carries the highest maintenance costs).

The monorepo will simplify this process. There are a couple of ways we can implement them in the monorepo architecture, and we’re still working on it, so I’ll leave it another report (next month?).

3. One deployment per Pull Request.

This is the (my) holy grail. I want us to be able to develop locally, push changes to the cloud, and have our infrastructure spin up the whole thing. Imagine working on a feature and being able to provide to the stakeholders with a unique URL for them to test, knowing that (1) that’s exactly how the system will behave once merged into production, and that (2) they can fiddle around as much as they want, as this environment is not shared with production nor a centralized staging environment. We’re not there yet, but that’s where we want to get.


The setup.

File structure.

We’re using lernajs alongside yarn workspaces to manage dependency installation and linking, and running tests for every service.

So far we have ported to the monorepo 5 services and 1 package.

CI Workflow.

We’re using CircleCI for testing and deployment. We use a custom .circleci/config.yml file, that calls lerna for tests, and calls a custom bash file for deployment (scripts/deploy.sh).

A successful `deploy` workflow. Note that services are deployed in parallel.

Dockerfile.

We’re basing our image on Alpine, doing a little PATH magic, and using yarn for dependencies.

As a courtesy for future devs, I’ve left a comment linking to an explanation on why Docker Layers rule 🚀

Dockerhub.

Our deploy script builds the docker image on the cloud (CircleCI) then pushes the images to Dockerhub. CircleCI has their own Docker Caching Layer so it skips redundant builds. Good job, CircleCI!

All image names were prefixed with `ygg` to avoid conflicts and overwrites during the transition.

Heroku.

We host our code on Heroku, and we were delighted to learn that Heroku has their own Docker Registry, so pushing an image is as simple as docker push registry.heroku.com/<repo>/<service>:<tag>.

The end result.

To level with you, after pushing the image, you still have to release it. Here’s the code

Error checking in bash FTW

Reviewing the first PR to the Monorepo.

In order to send this PR for review I was careful to keep the git history as clean as possible. As this is a new repo, I wanted master to be clean and everything to be on the first PR. This meant a lot of rebasing 🙂

Can you guess what “Batman” stands for?
Merging existing repos into the monorepo while dealing with open PRs was tricky, so these commit messages are not consistent with semantic types, but will be, moving forward.

And I even learned a new git trick while cleaning up the history, which I readily shared with the team (and with Twitter).

This is obvious in retrospect.

I’m stoked about the peace of mind the monorepo architecture provides to my team and me, and what it will yield us in terms of speed and quality.

Software Daily

Software Daily

 
Subscribe to Software Daily, a curated newsletter featuring the best and newest from the software engineering community.