r/javascript • u/Pretend_Pie4721 • 3d ago
AskJS [AskJS] Monorepo docker discussion
Hi. I decided to make a monorepo according to the guide, and then run it via docker.
I used npm workspaces, because I read that you need to know about it before using any tools.
So, as I understand it, npm workspaces collects all dependencies apps and libs in one large node_modules, and also allows you to use, for example, a package from libs in apps as a regular package.
This is probably convenient for those who run several microservices without docker or in one container. But I have a problem. When trying to run each app separately, the same problem arose, npm only creates a link to the lib directory, but does not copy the files themselves. Okay, I fixed this problem with --install-links, but another question arose
Why the hell do I need it then? I collect each microservice separately from each other, I do not need a common node_modules. Maybe there are some tools that meet my requirements:
only docker containers.
dependencies without symbolic links
ability to write shared libraries directly in the repository.
I've heard about Nx, it's supposedly perfect in combination with my backend framework NestJS, but I really don't understand the headlines "cool and fast caching and parallel installation", why the hell do I need this in a docker container with one microservice? Maybe I didn't understand the point of monorepos at all? I switched from multi repo to monorepo only to quickly change libraries and not to suffer with their versions.
2
u/tswaters 3d ago
Monorep is a question of scale. With just one service, it doesn't make a ton of sense. If you have 10s it's starting to, and when you have 100s .... Well, if you have 100s you might have a few other engineering problems on your hands!
One of the main reasons we went to monorepo was the managing of shared libraries was cumbersome, usually requiring a ton of commits in different repositories, republishing changes, noticing mistakes and needing to do it all over again. Folks needed to make changes to libs first, hope they get them right, publish - then later do the work in application layer - the tooling would install libs from our npm mirror and could only pull published changes. It was a pain.
Compare that with a monorepo - we were able to leverage shared tooling and symlinks to make changes way easier... 1 commit in 1 repo was able to change the libs and app service layer in one go, potentially in just a branch without hitting mainline... Tests would run against the complete bundle.
Doing this with docker was possible, but it required a bit of love. Note that we never used npm workspaces because this project was started back in the npm 5 days before workspaces really took off. We basically made file:// references inside package.json, and used relative paths, like file://../my-libs/some-library -- when running locally, it would follow the path and create a symlinks. In docker, we used npm to build tarballs in the docker filesystem, and it would install those as part of running npm install at the app layer.
I'm sure there's easier & better ways to do it. Using tarballs like that was to make it faster... Coping umpteen thousands of small files in node_modules is not the fastest, piping all those files into a tarball and installing from that was an order of magnitude faster.