Moving a Front-End Enterprise App into a Monorepo

There have been a lot of posts on the benefits of having all your organization’s code in a single repository, known as the monorepo. (See David R. MacIver’s post for a good summary.) Several popular projects, such as Babel and Plotly, follow this approach. After a lot of discussion, the front-end Banno team at Jack Henry decided to follow this approach for our web products.

The Problem

The Banno enterprise platform was written as multiple AngularJS apps. Each of these apps and their libraries were held in individual git repositories and published as npm packages. All of these dependencies were automatically pulled in through package management and wired up for the module bundler/loader to create the app “platform”.

This setup has a some significant disadvantages:

• Features and functionality are spread across multiple subprojects. This splits up the commits and the review process for what is conceptually a single change.
• Packages must then be version-bumped and propagated up through the dependency tree before the main platform can integrate and deploy them.
• Configurations and setup (e.g. tests, build tools) that are almost identical across repositories must be duplicated for each project.

Our goals were to reorganize the code to remove this friction:

• Let developers make a single pull request for a given feature/fix across the platform.
• Remove the staggered integration of changes up through the platform.

Finding the Right Fit

Before moving our code, we had to consider the trade-offs of a monorepo, possible tools to manage it, and how our workflow would change.

We used git submodules a bit in the past, but they are cumbersome to work with, and submodules are mainly for including code from repositories that must be separated. We also looked at management tools such as Lerna and Copybara, but they didn’t really fit our needs and were rather fiddly.

Npm, and package managers/registries in general, provide a great way to share and reuse modules. However, many of Banno’s libraries are private to its platform. A single repository removes the need to publish npm modules, and the resulting “dependency hell”. Instead, the module is required/imported using its local file name. Modules can still be published to an npm registry from the monorepo subfolder, but in general modules that are used outside the platform — especially open source — will remain as separate repositories.

The Banno platform ended up as a single git repository with subfolders for our apps, UI components, and libraries. The main benefit of this approach — consolidation — is also its biggest drawback. Repositories in GitHub are not designed for multiple projects, so filtering and watching issues/labels/milestones becomes trickier. Subscribing to a monorepo’s issues means subscribing to all subprojects, even if you’re only working on one. Combining multiple projects into a single repository also means the git history becomes busier. In the end, we decided this was an appropriate trade-off.

Implementation

Once you have the goal to move to a monorepo, how do you migrate your existing codebase? Here are the steps we took.

1. Move or clean up GitHub metadata for each repository that will be merged. This means deleting old, stale branches; merging any GitHub wiki pages into the new repository; synchronizing any .github templates; and copying/syncing any GitHub labels, milestones, etc. Move any issues that have not begun work; the GitHub issue mover makes this easy — although the UI is a bit wonky, it runs fine.
2. Begin copying the files from the repositories into the new one. You’ll probably want to use a method to preserve the git history; we used git subtree. In essence, it pulls commits from another repository and merges them into an arbitrary folder. We squashed the commits for the merges, to keep the main trunk of code readable. (We can always go back to the original repository for the old history.) You can even push commits back to the original repository with git subtree push, which is handy during migration.
3. Copy or update your CI/CD process as necessary to work with the subprojects’ new location in the monorepo.
4. Update any web hooks (such as with Slack or Waffle) to point to the new repository.
5. When ready, officially deprecate the old repositories.
6. Burn through all the remaining open issues. Refuse any new pull requests and issues to the old repositories — create them in the new monorepo from now on. Keep watch during the migration window to make sure everything is still building and running as it should.
7. Once all work has been cleaned up in the old repos and the monorepo is running smoothly, lock down the old repositories and declare success!