Updating dependencies

I’m speaking about XProc at Balisage next week. I’m going to be talking about debugging XProc pipelines and I’ll be using my implementation for demo purposes. To that end, I’ve been doing a little tidying up.

A couple of days ago, I saw mention of SchXslt2 version 1.4 and I made a note to check what version I’m using in XML Calabash. I thought I updated it fairly recently, but I also thought it was 1.3. I was right. So I upgraded to 1.4.4.

And because there was a lingering reference to FOP in my gradle.properties file, I went to see if that was up-to-date. Nope, that was 2.10 and the latest is 2.11. So I went to upgrade that.

You can see where this is going.

FOP 2.11 uses a newer version of pdfbox, and that’s used by…and that means…and before long, I just decided it was time to make sure I had the most recent dependencies across the board.

Luckily, I’ve already got a centralized list of the dependencies because I use them for the build, for the documentation, and for the info command. Still, I wasn’t about to go checking them all by hand. I poked about and discovered that Maven Central has a REST API that I can use to check the latest version of an artifact. Perfect.

Okay, so I want to take a list of artifacts, poke a REST API for each one, and keep track of the latest version. Easy peasy. Mock up a little XML vocabulary⊕Maybe you’re first thought wasn’t “use XML”, but I don’t think most of my readers will be terribly surprised to discover that it was mine. to hold the artifact references, write a little XSLT to call the REST API, parse the JSON response, and update the versions.

Cool, cool. I’ll be done in a few minutes.

The [expletive] search API returns incorrect results. It’s just flat out wrong.

Search for com.nwalsh:sinclude and the REST API reports the latest version as 5.3.0. I uploaded 5.4.1 two months ago, and that’s what you get from the regular web API.

I just don’t even.

Cool, cool. This is an opportunity, that’s what it is. It’s an opportunity to write a little XProc pipeline to solve the problem. What XProc can do (that XPath 3.x can’t) is load HTML pages. And the central repository web pages are very definitely HTML. (They’re the sort of awful, modern sludge that you see so often: packed full of encoded scripts, festooned with weird attributes; all-in-all more than 80k in the initial payload to display what amounts to less the 3k of text.)

Okay. Rant over. The sludge is machine generated so it’s regular and it’s passably well organized for what it is. And at least the answer is in the initial payload and it isn’t some [expletive] SPA that I have to interrogate with cx:selenium. (But I can, if you make me!)

The external dependency XML is processed by a pipeline that gets the most recent version of every dependency. Then we use XSLT to update the XML with the latest versions.

That gives me a new version of the dependency XML. I review that by hand. There are things I can’t upgrade (things that depend on more recent versions of Kotlin); there are things I don’t want to upgrade (to beta releases); and there are things I’m just choosing not to upgrade (the versions of the GraalVM polyglot libraries) because I don’t feel like testing them today.

One more XSLT script transforms the external dependency XML into the bit of Kotlin build script that the rest of the build system uses. I check that into the repository as a source file. This dependency management process is an as needed, ad hoc affair.

I had planned to be working on #115 today, but writing pipelines is fun!