My worst typo
D’oh!
I run MarkLogic in docker
containers with docker-compose
. For example:
nightly-1:
image: f23-runner:latest
container_name: nightly-1
hostname: nightly-1
networks:
app_net:
ipv4_address: 10.10.1.11
volumes:
- /space/docker/data/share:/share
- /space/docker/data/nightly-1:/var/opt/MarkLogic
stdin_open: true
tty: true
Using an external volume for /var/opt/MarkLogic
means both that MarkLogic configurations
survive
container restarts and that I can look at logs and configurations and such
from outside the container. Very handy.
In my build environment, MarkLogic runs as “my” process so I can clear out
the configuration data when I want to with a simple rm
. But when I run
a cluster of “nightly” containers, the MarkLogic process runs as root, so
I can’t simply delete the data.
I’m working on a particularly convuluted bit of configuration at the moment,
so I wanted to setup a couple of simple “save” and “restore” scripts. Easy peasy:
stop the containers, archive or restore the data/nightly-*
directories as
appropriate, restart the containers.
Let’s look a little more closely at the restore script.
Given that $NAME
contains the name of our saved configuration and
$LIST
contains the list of container names to restore, we run:
echo "Restoring archive: SAVE.$NAME.tar.gz"
for container in $LIST; do
sudo rm -rf $f/*
done
sudo tar zxf SAVE.$NAME.tar.gz
I pounded on ^C
as soon as I saw messages about not being able to
delete files in /dev/…
. And I had done a full backup on Tuesday.
And I wanted to die.
The actual lossage was quite small, a folder of docs in my home directory (none of which are likely to have changed since Tuesday) and, of course, most of the OS. Remarkably, the MarkLogic sources where I had spent many hours (since Tuesday!) tuning aforesaid convuluted configuration survived intact. Entirely by chance.
I really should investigate some sort of Time Machine-esque backup tool.
I have two sets of hard drives for my laptop; so part of the recovery was straightforward. Pull out the now borked drive, put in the other one, install a new OS on that, mount the old drive, and get on with recovery.
A couple of parts were quite a bit tricker than I expected.
-
My bootable USB key didn’t. Getting a bootable USB key for a ThinkPad from a Mac took longer than it should have, isohybrid what?
-
If you’re trying to mount an encrypted drive that used to be a boot drive, it’ll be in the same LVM volume group as your new encrypted boot drive. That means you can’t mount it. Booting off the USB, attaching the old drive, and renaming the volume group that it’s in worked for me.
-
I had to rebuild my node/npm/[expletive deleted -ed] environment. (
nave
seems to be working at the moment). -
It’s going to be ages before I’ve got everything reinstalled and I’ve recovered all the configuration files. At least in Unix, they’re usually just dot files in my home directory.
It took all day but I believe I’m back in operation.
A moment on the fingertips, hours and hours [and hours -ed] in the recovery.