Literate programming version of “t”

This weblog posting is an attempt to make something useful out of a currently failed experiment. It’s also an opportunity for someone with a clever idea about how to make a lambda out of an async function or otherwise work around the globals to send me a pointer. ☺

I deal with servers all over the world: at home, at work, at my ISP, at AWS, etc. Several times a day, I want to open up a shell window on one of those remote systems.

Suppose I open up a root shell on my local machine, a remote shell at work, and a remote shell on AWS. Now I want to delete some files, so I type rm -rf …. The jokes almost write themselves, don’t they?

I have a couple of large monitors and a high-resolution color display. Rather than using ssh in my ordinary shell window, I can configure different terminal windows, with different color schemes: matrix green on black, that’s my root window; MarkLogic red background, that’s my work window; an Amazon orange background, that’s my AWS window; etc.

When Docker enters the picture, it gets a little more complicated. What you want to do in the Docker case isn’t ssh but docker exec. Except, to do that you have to know the ID of the container and those change everytime you start a new one.

I quickly got tired of that, so I wrote dps a shell script to simplify my interations with docker. And then I realized that I could integrate that with my “start a terminal” script and t was born.

• t → open a new terminal window on my local system.
• t presentation → open a new terminal window on my local system with large fonts and high contrast.
• t marklogic → open a terminal at work.
• t aws → open a terminal at AWS.
• t pihole → open a terminal on the RaspberryPi running Pi-hole.
• t openwrt → open a terminal on my router.
• t builder → open a terminal on the Docker containter “builder”.

And for good measure you can specify a command to run instead of just logging in.

• t -c "/share/ml-compile b9_0" builder → build the b9_0 branch of MarkLogic server on the “builder” container

The t script was initially written in Perl and ran on both Linux and the Mac. I recently convert it to Python in order to take advantage of the Python APIs for ITerm2, my prefered MacOS terminal.

It starts off as a completely standard Python3 program.

#!/usr/bin/env python3

"""Creates a new iTerm window to execute a command. Interrogates
Docker and SSH configurations to determine a reasonable default command.
Profiles can be configured for different environments.
"""

import re
import os
import sys
import json
import argparse
import subprocess
from json import JSONDecodeError

The iterm2 and AppKit modules provide the ability to talk to iTerm2.

import iterm2
import AppKit

Putting HOME in a variable is just a convenience.

HOME = os.environ["HOME"]

The asynchronous function (runiterm, below) that actually makes the terminal needs to know what command to run and what profile to use, so these are globals.

ZSHCOMMAND = None
PROFILE = None

Global variables are frowned upon, but I can’t work out any other way to do it:

• I can’t pass them as arguments to the function, because the function is part of the iTerm2 API.
• I can’t construct the function as a lambda because (AFAICT), you can’t make an async lambda.

If there’s something I’ve overlooked, I’d love to hear it.

The runiniterm function itself is the one that actually interacts with the iTerm2 API. It gets an asynchronous connection to the running application and uses the API to make a new terminal with the desired profile.

async def runiterm(connection):
    """Runs ZSHCOMMAND in iTerm2. Shame about using the global."""
    app = await iterm2.async_get_app(connection)

    # Foreground the app
    await app.async_activate()

    if PROFILE is not None:
        all_profiles = await iterm2.PartialProfile.async_query(connection)
        for prof in all_profiles:
            if prof.name == PROFILE:
                await prof.async_make_default()

    if ZSHCOMMAND is None:
        zsh = "/usr/local/bin/zsh --login -i"
    else:
        zsh = '/usr/local/bin/zsh --login -i -c "%s"' % ZSHCOMMAND
    await iterm2.Window.async_create(connection, command=zsh)

There are (currently) four places where we can look to find a mapping between the specified terminal and the underlying commands.

To start with, there’s a configuration file ~/etc/t-profiles.json for just this purpose:

{
  "default": {},
  "root": {
    "profile": "Root",
    "command": "sudo su -"
  },
  "openwrt": {
    "profile": "Root",
    "command": "ssh openwrt"
  },
  "aws": {
    "profile": "AWS",
    "command": "ssh aws"
  },
  "dunsinane": {
    "profile": "Dunsinane",
    "command": "ssh dunsinane"
  },
  "presentation": {
    "profile": "Presentation"
  }
}

Each key in the top-level map is a possible terminal name. The profile identifes the iTerm2 profile to use. “Root” is matrix green-on-black, “Presentation” is large and high-contrast, etc.

The check_profiles function looks in the configuration file.

def check_profiles(term, command):
    """Search for the specified term in the global profiles."""
    name = None
    cmd = None

    try:
        with open("%s/etc/t-profiles.json" % HOME) as profile:
            profiles = json.load(profile)
    except FileNotFoundError:
        profiles = []
    except JSONDecodeError:
        profiles = []

    if term in profiles:
        if command is None and "command" in profiles[term]:
            cmd = profiles[term]["command"]
        if "profile" in profiles[term]:
            name = profiles[term]["profile"]

    return (name, cmd)

Another place we could look is among the running Docker containers. The check_docker function looks there.

def check_docker(term, command):
    """Search for the specified term in the running Docker containers.
    This function relies on 'dps' a wrapper script that simplifies
    interaction with 'docker ps' and friends"""
    name = None
    cmd = None

    # Maybe a running container?
    proc = subprocess.run(args=["dps", "id", term], check=False, capture_output=True)
    if proc.returncode != 0:
        print("Ambigous container id: %s" % term, file=sys.stderr)
        sys.exit(1)
    cid = proc.stdout.decode("utf-8").strip()
    if cid != "":
        name = "Docker"
        if command is None:
            cmd = "dps login %s" % cid
        else:
            cmd = "dps exec %s %s" % (cid, command)

    return (name, cmd)

My dps script (not shown) will search for containers by container name and image name. If we find a container name that matches, then we construct an appropriate command.

Another place to look is in my ~/.ssh/config file. You can use the SSH configuration file to simplify SSH. For example, consider the following fragment of my SSH config (fuzzed for privacy):

Host aws
  Hostname ec2-XX-XX-XX.us-XXX.compute.amazonaws.com
  User ec2-user
  IdentityFile ~/.ssh/AWS-EC2.pem

If I run ssh aws, this configuration will connect to the correct host, as the correct user, with the correct key. I no longer have to type out the long form.

The check_ssh function looks for a matching SSH configuration.

def check_ssh(term, command):
    """Search for the specified term in my ssh configuration."""
    name = None
    cmd = None

    # Maybe ssh?
    hostre = re.compile(r"^Host\s+%s" % term)
    with open("%s/.ssh/config" % HOME, "r") as config:
        for line in config:
            if hostre.match(line):
                name = "SSH"
                cmd = "ssh %s" % term
                if command is not None:
                    cmd = cmd + " " + command

    return (name, cmd)

Last, but not least, we can look in /etc/hosts. That’s what check_hosts does.

def check_hosts(term, command):
    """Search for the specified term in /etc/hosts."""
    name = None
    cmd = None

    hostre = re.compile(r"^\d+\.\d+\.\d+\.\d+\s+%s" % term)
    with open("/etc/hosts", "r") as hosts:
        for line in hosts:
            if hostre.match(line):
                name = "SSH"
                cmd = "ssh %s" % term
                if command is not None:
                    cmd = cmd + " " + command

    return (name, cmd)

There’s a utility function to get command line arguments. Mostly because Pylint barked at me for having a main function that was too long.

def parse_command_line():
    """Get the terminal selection and optional command from the command line."""
    parser = argparse.ArgumentParser()
    parser.add_argument("--command", "-c", action="store", help="Command to run")

Note that we specify default as the default profile. That’ll come up later.

parser.add_argument(
    "profile", action="store", nargs="?", default="default", help="Profile name"
)
args = vars(parser.parse_args())
term = args["profile"]
command = args["command"]
return (term, command)

Finally, we get to the main function.

def main():
    """Run command in an iTerm window with profile"""

I have to use the aforementioned globals, so I’m telling pylint explicitly not to ding me for them.

# pylint: disable=global-statement
global ZSHCOMMAND  # hack
global PROFILE  # hack

Next we get the initial terminal and command.

(term, command) = parse_command_line()

Then we look in the t profiles, docker containers, SSH, and hosts for that terminal. First one wins.

(name, cmd) = check_profiles(term, command)
if name is None:
    (name, cmd) = check_docker(term, command)
if name is None:
    (name, cmd) = check_ssh(term, command)
if name is None:
    (name, cmd) = check_hosts(term, command)

If we didn’t find the terminal, that’s ok if it’s the special profile default. In that case, we’ll just make a new terminal on the local host.

if name is None:
    if term == "default":
        pass
    else:
        print("Error: no such profile: %s" % term, file=sys.stderr)
        sys.exit(1)

If we got this far, we’ve figured out what to do. What to do is, move ourselves back to the home directory (for consistency) and use the iTerm2 API to spin up the shell.

os.chdir(HOME)

# Launch the app with iTerm
# pylint: disable=no-member
AppKit.NSWorkspace.sharedWorkspace().launchApplication_("iTerm2")

I don’t really understand how AppKit works. It’s true, as pylint warns me, that there appears to be no member named NSWorkspace, but it does, in fact, work, so I tell pylint to leave me alone.

PROFILE = name
ZSHCOMMAND = cmd
iterm2.run_until_complete(runiterm, True)

Finally, we end with the standard trick for allowing a module to be either run or imported. Not that it’s designed to be useful as a module, mind you.

if __name__ == "__main__":
    main()

That’s the script.

The pedantic among you may be itching to point out that what this post describes, and the way “tangle” in org-mode works, is not literate programming in the Knuthian sense as it offers no variable references, no reordering of blocks, and other “literate” features.

I understand those things. And if you’ve read this far, you will already have experienced the weakenesses that those features would mitigate: there are awkward forward references in several places, and generally the exposition would be improved by some reordering.

I’m still interested in this feature of org-mode, limitations nowithstanding.