so

Testing NodeJS programs

Volume 5, Issue 3; 12 Feb 2021

In the NodeJS world, how do you manage dependencies during development and testing? (Resolved.)

TL;DR In the scenario described below, what I had failed to do (and I don’t know how I was supposed to guess that I had to do this), was run npm install in each of the package directories before running npm install in the testing environment.

Let’s say I’m building a Node.js application. I have a package, “some-library”, that I can npm install and I have another package, “some-utility”, that uses my library package and provides a command line utility wrapper for it.

This is all fine. In the package.json file for “some-utility”, I say that it depends on “some-library”:

{
  "dependencies": {
    "some-library": "^1.0.0"
  }
}

As long as I’ve published the library and the utility in the npm registry, I can setup a package.json file for my test environment, run npm install, and everything works.

Now let’s say I’m working on version 2cough “2.1” of my library and utility. I’ve found a couple of different ways to get the development version of the library installed in the testing environment. The most straightforward seems to be to change the package file in the test environment so that the library dependency is a relative file reference to the development package.

But how do I get the utility to install?

If I say that the utility depends on “^2.0.0” of the library, it fails to install the utility because it can’t find the dependent library (in the npm registry, I suppose). This is frustrating since it happens even if I’ve already successfully installed version “2.0.0” of the library!

If I take the “some-library” dependency out of the utility, then it’ll install, but attempting to require the library fails. This is frustrating since it happens even if I’ve already successfully installed version “2.0.0” of the library!

(Apparently, there’s more book keeping going on here about what packages depend on what and what mechanisms can be used to resolve attempts to require them.)

If I use relative paths from “some-utility” to “some-library” and then use relative paths from my testing environment to “some-utility” and “some-library”, it falls over for some other completely mystifying reason:

npm WARN tar ENOENT: no such file or directory, \
  open '/Long/Path/Elided/node_modules/.staging/axios-0c7f00c7/dist/axios.map'
npm ERR! cb() never called!

npm ERR! This is an error with npm itself. Please report this error at:
npm ERR!     <https://npm.community>

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/ndw/.npm/_logs/2021-02-11T18_17_09_291Z-debug.log

And for good measure, …/dist/axios.map does exist!

This is just weirdly complicated for what I would have imagined was a fairly common occurrence.

The closest I’ve come to a solution is Yalc, but that has complications as well because it’s yet another third party dependency in the build and it’s complicated to make sure that the yalc-installed versions are correctly updated with each source code change and that the resulting packages are correctly installed in the test environment each time.

Someone, please tell me I’m overlooking something completely obvious.