so…Norm's musings. Make of them what you will. WalshXML Unicode 1.25

I’ve updated my Emacs package for inserting arbitrary Unicode characters.

Volume 5, Issue 22; 29 Aug 2021

I’ve updated my Emacs package for inserting arbitrary Unicode characters.

Back in July, I made a few updates to my xmlunicode package. One of the changes I made was to delay loading the XML Unicode database until the first time it was needed.

That shaves a couple of seconds off startup, but just moves the delay to the first time you attempt to use xmlunicode-character-insert. (Only the first time, I hasten to add, the data structure is cached after that.)

A few days later, I must have restarted Emacs for some reason and the first time I hit C-t u everything just sort of stopped. Not for a long time, but long enough for me to wonder what the heck was going on before the character list popped up.

I realized I should have put in a “loading” message so that the delay wasn’t mysterious.

I’ve done that now in version 1.25. And updated to version 14.0.0d13 of the Unicode character list.

Dash docset updates

Updates and additions to the Dash docsets I mentioned recently.

Volume 4, Issue 26; 18 Apr 2020

Updates and additions to the Dash docsets I mentioned recently.

A week ago or so, I described some work I’d done to make the XML specifications that I use frequently available in Dash.

Since then, I’ve been tinkering in the evenings, mostly in the time between what would reasonably be considered the end of the day and the beginning of my next meeting. (California, as I observed, is a lot of hours west.)

The latest updates are in the same place:

What’s changed?

  1. I decided to “chunk” the files. Dash is designed to be an API inspector, not a general purpose document reader. Mostly, I think that means it works best for targeted searches: how does the p:cast-content-type step work?; what goes in the picture string for fn:format-dateTime()?, etc. To that end, smaller faster pages seemed better.
  2. I stripped out script tags and a few navigation links that don’t make sense in this context.
  3. I added an “Annotation” index entry that includes details about the Dash conversion, including the version of the document. It’s weird that Dash doesn’t seem to have a standard place for version information in the docset.
  4. I sorted out the formatting issues with DocBook and added DocBook 5.2: The Definitive Guide.
  5. I added a few new XML specifications (XML 1.0, Namespaces 1.0, XML Base, XInclude, and XML Schema parts 1 and 2).
  6. I renamed the .tgz files to .docset.tgz because apparently that makes one of the Windows Dash clients happier.
  7. I’ve tried to make good use of index entry types. This works better in some specifications than others. I haven’t tracked down and identified every single anchor as an index entry. That said, if you think I missed a useful one, let me know.

Please try them out and report problems you encounter. I have one report of the XML Schema docsets not working correctly in the Windows client, but they look fine to me. Someone also asked about contributing these to the Dash. Yes, I’m happy to do that, but I’d like to wait a bit and see if any bugs are reported.

In other news…

I was delighted to discover an Emacs integration

Marginal note:The dash-docs repository suggests that it doesn’t work on MacOS, but it works just fine for me with Emacs 27.0.91. (I had to build my own Emacs to get SVG support and I figured I’d go ahead and move up to 27.)

for Dash:! Here’s my current config:

(use-package dash-docs)
(use-package helm-dash)
(require 'dash-docs)
(require 'helm-dash)
(setq browse-url-browser-function 'eww-browse-url)

(defun xml-doc ()
  (setq-local dash-docs-docsets
              '("defguide52" "file-30" "mail-30" "os-30" "paged-media-30"
                "rdf-30" "steps-30" "text-30" "validation-30" "xinclude"
                "xml5e" "xmlbase" "xmlns" "xmlschema11-1" "xmlschema11-2"
                "xpath-31" "xpath-datamodel-31" "xpath-functions-31"
                "xproc-30" "xquery-31" "xslt-30"
                "xslt-xquery-serialization-31" "xvrl")))
(add-hook 'nxml-mode-hook 'xml-doc)

(defun python-doc ()
  (setq-local dash-docs-docsets '("Python_3")))
(add-hook 'python-mode-hook 'python-doc)

;; etc.

It works really well!

At any point, I can hit my “dash docs” key binding, enter the first few characters of the API I’m interested in, and quickly get a menu of links to relevant docs.

Dash Docs

If I’m editing XML, the doc choices come from the XML docsets. If I’m editing Python, the doc choices come from the Python docs, etc.

Unfortunately, it looks like Dash has started saving the actual web pages in a compressed archive (a tarix.tgz file) so newer docsets don’t work in Emacs. Emacs could be made to support that, but it’s not going to make its way onto my todo list. I did quick experiment and it seems like unpacking the archive also works, so I’ll probably just do that.

Docker explorer

A small hack combining Emacs, docker, and MarkLogic log files.

Volume 4, Issue 19; 05 Mar 2020

A small hack combining Emacs, docker, and MarkLogic log files.

In my day job, I work on MarkLogic server (and other things). For years, I’ve been doing this by running the server in a Docker container (or often a small cluster of such containers). This isolates my development environment from changes in my laptop (not just OS upgrades, but changing the whole OS).

A typical session goes like this: start a container, start the server, start working on some code, open a shell window, exec into the container, tail -f a log file, go on about my work. This is fine until some bit of code squirts a whole lot of lines into the log file, then I’m scrolling around looking for bits. If I need to search the log, maybe I hack about with cat and grep or maybe I just bring in the big guns and open the log file in Emacs.

Hold that thought.

A few days ago, I saw the release announcement for a new version of transient. Transient is part of the magic that is Magit. I’m not a hard core magit user, I’ve probably barely scratched the surface, but I love the things it does for me.

A quick skim of the transient project left me none the wiser this time than it had last time. But sometime shortly after, probably in r/emacs, I found a link to Adrien Brochard’s talk about integrating transient into a workflow for managing Kubernetes log files. I’m not usually a fan of video tutorials, but I was curious enough about transient to check it out and it is excellent. Adrien does a fantastic job of leading his audience through both how he approached the problem and how he tackled it. I recommend it highly and if you have to choose between reading the rest of this page and watching that video, watch the video. Go! Shoo! I’ve got nothing significant to add.

I’m sort of serious about that last part.

The log file monitoring example that Adrien describes is the same in all but detail to the situation I described above. And reworking it to solve my problem was both an obvious way to learn a little more about transient and an obvious way to improve my workflow.

The project is docker explorer. Yes, it’s about reading MarkLogic log files, but the real beauty here is how transient connects the idea of building a UX with Emacs.

Type M-x docker-explorer and you’ll get a menu of your docker containers.

Container list

Select a running container and press l to open the logs window. (There’s nothing in this first selection that’s related to transient, but I’m already thinking about adding other commands that would use it.)

Logs list

Finally, pressing t brings up the tail menu:

Log tail

What transient is providing here is a nice, declarative interface to this functionality. The relevant bit of code is this:

(define-transient-command dxp--transient-tail-ml ()
  "Docker Explorer Tail command"
   ("-f" "Follow" "-f")
   ("t" "Tail" dxp--tail-ml-function)])

The “​-f​” flag (which creates a process that continues to tail the log file as it changes) is just a flag. The “​-n​” flag takes an argument and it’s declared with dxp-transient:--lines:

(define-infix-argument dxp-transient:--lines ()
  "Transient support for the lines argument."
  :description "Number of lines"
  :class 'transient-option
  :shortarg "-n"
  :argument "-n ")

There’s lots here I’m still digesting, and lots I frankly don’t understand, but it’s enough to see first hand how cleanly the declarative interface works. I’ll be coming back to explore this in more detail when I have the time.

And bonus: now, instead of fiddling with a shell window and trying to find things with grep, the whole searchable, evolving log file is in an Emacs buffer three or four keystrokes away. Win!

Not bad for an evening’s hacking on the sofa and ~150 lines of Emacs lisp.

My .emacs

Configuring emacs is part art, part science. I’m carrying around customizations that date back more than 20 years.

Volume 4, Issue 16; 29 Feb 2020

Configuring emacs is part art, part science. I’m carrying around customizations that date back more than 20 years.

This posting is the “woven” version of my Emacs configuration. It occurred to me that since I can publish Org mode files as weblog postings, I could post it directly. That amused me. It’ll be more useful to you from the repository. One obvious deficiency

Marginal note:Also, this posting contains a lot of third-level headings. Those are formatted as “run in” heads which usually work fine because I write the prose so that they do. Some of them look a bit odd in this posting; that’s another deficiency I’m not motivated to fix.

in this posting is that many of the code fragment are too wide. Given that the weblog is secondary and actually understanding the code in my Org file is primary, I’m not motivated to try to fix that.

The relevant point is, I maintain my Emacs configuration in a documented Org-mode file and that feels good.

This document represents and attempt to both document and clean up my configuration. This is, obviously, an Org-mode file. The code snippets it contains are woven together with their documentation (prose like this) and tangled together into my ~/.emacs.d/init.el file by Org-Babel.

One consequence of this approach is that I’m free to document code fragments in any order I want. Mostly, this file documents things from top-to-bottom, but the actual order is determined by the tangle, not the weave, as it were.

There’s still room for improvement:

  • A few big chunks of this file are still mostly verbatim copies from my old configuration file with little by way of new documentation.
  • There’s still a bunch of undocumented configuration in libraries in ~/.emacs.d/personal/*.el that would benefit from greater documentation.

The sincerest form of flattery

Insert witty quote about imitation, theft, and plagiarism here. Many of the ideas in this file are stolen from inspired by the works of others:

(Note to self: update this list, there are surely others.)


Lexical binding · Use lexical-binding. Why?

;;; init.el --- -*- lexical-binding: t -*-

Early Init · Emacs 27 introduces early-init.el, which is run before init.el, before package and UI initialization happens.

Compatibility with Emacs 26

Ensure emacs-version>=26, manually require early-init configurations if emacs-version<27.

(cond ((version< emacs-version "26.1")
       (warn "M-EMACS requires Emacs 26.1 and above!"))
      ((let* ((early-init-f (expand-file-name "early-init.el" user-emacs-directory))
              (early-init-do-not-edit-d (expand-file-name "early-init-do-not-edit/" user-emacs-directory))
              (early-init-do-not-edit-f (expand-file-name "early-init.el" early-init-do-not-edit-d)))
         (and (version< emacs-version "27")
              (or (not (file-exists-p early-init-do-not-edit-f))
                  (file-newer-than-file-p early-init-f early-init-do-not-edit-f)))
         (make-directory early-init-do-not-edit-d t)
         (copy-file early-init-f early-init-do-not-edit-f t t t t)
         (add-to-list 'load-path early-init-do-not-edit-d)
         (require 'early-init))))

Defer garbage collection

Defer garbage collection further back in the startup process, according to hlissner:

The GC eats up quite a bit of time, easily doubling startup time. The trick is to turn up the memory threshold as early as possible.

(setq gc-cons-threshold 100000000)

Disable package-enable-at-startup

Package initialize occurs automatically, before user-init-file is loaded, but after early-init-file. We handle package initialization, so we must prevent Emacs from doing it early!

(setq package-enable-at-startup nil)

Unset file-name-handler-alist

Every file opened and loaded by Emacs will run through this list to check for a proper handler for the file, but during startup, it won’t need any of them.

(defvar file-name-handler-alist-original file-name-handler-alist)
(setq file-name-handler-alist nil)

Disable site-run-file

(setq site-run-file nil)

Disable unnecessary interfaces

It will be faster to disable them here before they've been initialized.

;;(menu-bar-mode -1)
(unless (and (display-graphic-p) (eq system-type 'darwin))
  (push '(menu-bar-lines . 0) default-frame-alist))
(push '(tool-bar-lines . 0) default-frame-alist)
(push '(vertical-scroll-bars) default-frame-alist)

Garbage Collection

Set gc-cons-threshold smaller for interactive use

A large gc-cons-threshold may cause freezing and stuttering during long-term interactive use.

(defvar better-gc-cons-threshold 67108864 ; 64mb
  "The default value to use for `gc-cons-threshold'.
If you experience freezing, decrease this. If you experience stuttering, increase this.")

(add-hook 'emacs-startup-hook
          (lambda ()
            (setq gc-cons-threshold better-gc-cons-threshold)
            (setq file-name-handler-alist file-name-handler-alist-original)
            (makunbound 'file-name-handler-alist-original)))

Garbage collect when emacs is out of focus

Note: Since Emacs 27.1, focus-out-hook is obsolete.

(add-hook 'emacs-startup-hook
          (lambda ()
            (if (boundp 'after-focus-change-function)
                (add-function :after after-focus-change-function
                              (lambda ()
                                (unless (frame-focus-state)
              (add-hook 'after-focus-change-function 'garbage-collect))

Avoid garbage collection when using minibuffer

(defun gc-minibuffer-setup-hook ()
  (setq gc-cons-threshold (* better-gc-cons-threshold 2)))

(defun gc-minibuffer-exit-hook ()
  (setq gc-cons-threshold better-gc-cons-threshold))

(add-hook 'minibuffer-setup-hook #'gc-minibuffer-setup-hook)
(add-hook 'minibuffer-exit-hook #'gc-minibuffer-exit-hook)))

Use the garbage collector magic hack

The garbage collector magic hack.

   :type git
   :host github
   :repo "emacsmirror/gcmh"))
(require 'gcmh)
(gcmh-mode 1)

Load Path · I’m storing some functions in ~/.emacs.d/personal, so make sure that gets added to the load path. For good measure, add its subdirectories as well.

(defun update-to-load-path (folder)
  "Update FOLDER and its subdirectories to `load-path'."
  (let ((base folder))
    (unless (member base load-path)
      (add-to-list 'load-path base))
    (dolist (f (directory-files base))
      (let ((name (concat base "/" f)))
        (when (and (file-directory-p name)
                   (not (equal f ".."))
                   (not (equal f ".")))
          (unless (member base load-path)
            (add-to-list 'load-path name)))))))

(update-to-load-path (expand-file-name "personal" user-emacs-directory))

Define constants · I nicked these from somewhere. They seem like a good idea, but I’m not actually using most of them just at the moment.

(defconst *sys/gui*
  "Are we running on a GUI Emacs?")

(defconst *sys/win32*
  (eq system-type 'windows-nt)
  "Are we running on a WinTel system?")

(defconst *sys/linux*
  (eq system-type 'gnu/linux)
  "Are we running on a GNU/Linux system?")

(defconst *sys/mac*
  (eq system-type 'darwin)
  "Are we running on a Mac system?")

(defconst *sys/root*
  (string-equal "root" (getenv "USER"))
  "Are you a ROOT user?")

(defconst *rg*
  (executable-find "rg")
  "Do we have ripgrep?")

(defconst *python*
  (executable-find "python")
  "Do we have python?")

(defconst *python3*
  (executable-find "python3")
  "Do we have python3?")

(defconst *mvn*
  (executable-find "mvn")
  "Do we have Maven?")

(defconst *gcc*
  (executable-find "gcc")
  "Do we have gcc?")

(defconst *git*
  (executable-find "git")
  "Do we have git?")

(defconst *pdflatex*
  (executable-find "pdflatex")
  "Do we have pdflatex?")

Where is our custom file?

I’m not really a fan of the custom framework; I’d rather edit the sources and save my configurations by hand. But some packages use it, so let’s get this out of the way early.

;;; Tell custom to put its crap somewhere else, but load it early
(setq custom-file (concat user-emacs-directory "custom.el"))
(load custom-file 'noerror)

Remove built-in Org

I want to install a version of Org-Mode that’s more recent than the version that ships with Emacs. To do that safely, I first move the built-in version out of my class path.

;; Remove the built-in version of Org from the load-path
(require 'cl-seq)
(setq load-path
       (lambda (x)
         (string-match-p "org$" x))

Load the package manager

I’ve been using straight.el for a while and I really like it.

;; Bootstrap straight.el
(defvar bootstrap-version)
(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
         'silent 'inhibit-cookies)
      (goto-char (point-max))
  (load bootstrap-file nil 'nomessage))

Load up the straight.el version of use-package.

;; Load the straight.el version of use-package
(defvar straight-use-package-by-default)
(straight-use-package 'use-package)
;; Tell straight to use use-package by default
(setq straight-use-package-by-default t)

;; Tell straight to use ssh by default, instead of https.
(setq straight-vc-git-default-protocol 'ssh)

System configuration

Configure the keyboard · What kind of a system is this? Do the bare requirements necessary to make the keyboard usable.

  (if *sys/mac*
        (setq mac-command-modifier 'meta) ; command = Meta
        (setq mac-option-modifier 'super) ; (left) option = Super
        (setq mac-control-modifier 'control) ; control = Control
        (setq mac-right-option-modifier 'hyper) ; (right) opton = Hyper
        (setq x-select-enable-clipboard 't)))

Load my custom functions · Load my custom functions.

;; Some custom functions.
(require 'ndw-defuns)

(define-key global-map "\M-j" 'ndw/fill-paragraph)

Grab bag of my general settings · This needs to be cleaned up.

;; This is me.
(setq user-mail-address "")
(setq user-full-name "Norman Tovey-Walsh")
(setq add-log-mailing-address "")

; Allow some things that emacs would otherwise confirm.
(put 'eval-expression  'disabled nil)
(put 'downcase-region  'disabled nil)
(put 'upcase-region    'disabled nil)
(put 'narrow-to-region 'disabled nil)
(put 'set-goal-column  'disabled nil)

;; No flashing, please.
(setq visible-bell nil)

;; Also, I’ve seen the startup message
(setq inhibit-startup-message t)

;; Show me errors
(setq debug-on-error t)

;; Show me more log messages
(setq message-log-max 500)

;; WTF? If I don't define this, I get weird warnings when byte compiling
(setq warning-suppress-types nil)

(unless *sys/win32*
  (set-selection-coding-system 'utf-8)
  (prefer-coding-system 'utf-8)
  (set-language-environment "UTF-8")
  (set-default-coding-systems 'utf-8)
  (set-terminal-coding-system 'utf-8)
  (set-keyboard-coding-system 'utf-8)
  (setq locale-coding-system 'utf-8))
;; Treat clipboard input as UTF-8 string first; compound text next, etc.
(when *sys/gui*
  (setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING)))

(setq electric-indent-mode nil)
(setq case-fold-search t)
; Character folding in searches!
(if (boundp 'search-default-mode)
    (setq search-default-mode 'char-fold-to-regexp))

(setq line-number-mode t)
(setq auto-save-interval 2048)
(setq blink-matching-paren-distance 100000)

;; Updates from [[][alhassy]].
;; Move auto save and backup files out of the way
(setq backup-directory-alist '(("." . "~/.emacs.d/backups")))
;; Silently delete execess backup versions
(setq delete-old-versions t)
;; Only keep the last 100 backups of a file.
(setq kept-old-versions 000)
;; Even version controlled files get to be backed up.
(setq vc-make-backup-files t)
;; Use version numbers for backup files.
(setq version-control t)

;; Make emacs backup every time I save.
(defun ndw/force-backup-of-buffer ()
  "Lie to Emacs, telling it the current buffer has yet to be backed up."
  (setq buffer-backed-up nil))
(add-hook 'before-save-hook  'ndw/force-backup-of-buffer)

;; Follow symlinks automatically
(setq vc-follow-symlinks t)

;; Ignore .rtf files
;; (Does anyone remember .rtf files? I don’t think I’ve seen
;; one since the last millenium.)
(setq completion-ignored-extensions
      (append completion-ignored-extensions '(".rtf")))

;; Use python3
(setq python-shell-interpreter "python3"
      python-shell-interpreter-args "-i")

;; Location things
(setq calendar-latitude 30.3102244)
(setq calendar-longitude -97.7528019)
(setq calendar-location-name "Austin, TX")

;; Bookmarks (which I never think to use)
(autoload 'create-bookmark "bookmark" "Bookmark utility" t)
(autoload 'locate-bookmark "bookmark" "Boomkark utility" t)
(autoload 'query-delete-bookmarks "bookmark" "Boomkark utility" t)

;; Emerge
(autoload 'emerge-files "emerge" "File merging utility" t)
(autoload 'emerge-files-with-ancestor "emerge" "File merging utility" t)

;; Activate transparent GnuPG encryption.
(require 'epa-file)

;; Save minibuffer history
(require 'savehist)
(savehist-mode t)

;; Save the place in files
(require 'saveplace)
(setq-default save-place t)
(setq save-place-file "~/.emacs.d/saved-places")

;; Setup authinfo
(require 'auth-source)
(if (file-exists-p "~/.authinfo.gpg")
    (setq auth-sources '((:source "~/.authinfo.gpg" :host t :protocol t)))
    (setq auth-sources '((:source "~/.authinfo" :host t :protocol t))))

Immortal buffers · Some buffers should never be deleted. Even if you type C-x k in them.

;; Make some buffers immortal
(defun ndw-immortal-buffers ()
  (if (or (eq (current-buffer) (get-buffer "*scratch*"))
          (eq (current-buffer) (get-buffer "*Messages*")))
      (progn (bury-buffer)

(add-hook 'kill-buffer-query-functions 'ndw-immortal-buffers)

Prettify things · These settings “prettify” some things in the buffer. For example, the string “l-a-m-b-d-a” is replaced by the “λ” character. This has no effect on the underlying data, it only effects the presentation. This is a little bit like using a programming font with ligatures, an approach not universally admired.

I’m on the fence about it. In some contexts, I think it’s fine. In others, it really does look a bit odd.

(global-prettify-symbols-mode 1)

(defun add-pretty-lambda ()
  "Make some word or string show as pretty Unicode symbols.
See for more."
  (setq prettify-symbols-alist
          ("lambda" . ?λ)
          ("->" . ?⟶)
          ("<=" . ?≤)
          (">=" . ?≥)
          ("#+BEGIN_SRC" . ?✎)
          ("#+END_SRC"    . ?□)
          ("#+BEGIN_EXAMPLE" . (?ℰ (Br . Bl) ?⇒)) ;; ℰ⇒
          ("#+END_EXAMPLE"    . ?⇐)               ;; ⇐
          ("#+BEGIN_QUOTE" . (?𝒬 (Br . Bl) ?⇒))   ;; 𝒬⇒
          ("#+END_QUOTE"    . ?⇐)                 ;; ⇐

;; Alternatively, rendering begin/end src as icons can be improved:

(add-hook 'text-mode-hook 'add-pretty-lambda)
(add-hook 'prog-mode-hook 'add-pretty-lambda)
(add-hook 'org-mode-hook 'add-pretty-lambda)

Whitespace · I’m fussy about whitespace.

(require 'whitespace)
;; 17 October 2019, removed "face" here because it makes colored backgrounds
;; in other modes, such as Org and Markdown, ugly and distracting.
;; 3 November 2019, removed trailing because too many menus have it
(setq whitespace-style
      '(tabs trailing spaces space-before-tab newline space-after-tab space-mark tab-mark))
(setq whitespace-display-mappings
;        (space-mark   ?\    [?\x2423]   [?·] [?\ ])  ; space
        (space-mark   ?\xA0 [?\xA4]     [?_])        ; hard space
(setq-default show-trailing-whitespace nil)

(setq-default fill-column 70)
(setq sentence-end-double-space nil)
(setq-default indicate-empty-lines t)
(setq-default indent-tabs-mode nil)

Custom key bindings · I have a set of custom keybindings. There’s a school of thought that says custom bindings are bad because they limit your ability to be productive on an Emacs not configured your way. That almost never happens to me, so I’m not going to worry about it.

Control T bindings

I bind a bunch of random things to “​C-t​” because I learned Emacs in graduate school and this is what my advisor did. I do not assert that I know and remember all of these bindings all of the time.

  ;; The principal of the lab where I was introduced to Emacs
  ;; installed extensions on C-t. So I do too.
  (defvar ctl-t-map nil "Norm's extensions key map.")
    (setq ctl-t-map (lookup-key global-map "\C-t"))
    (if (not (keymapp ctl-t-map))
        (setq ctl-t-map (make-sparse-keymap)))
    (define-key ctl-t-map "AT"   'ndw/agenda-template)
    (define-key ctl-t-map "AM"   'ndw/minutes-template)
    (define-key ctl-t-map "RC"   'clear-rectangle)
    (define-key ctl-t-map "RD"   'delete-rectangle)
    (define-key ctl-t-map "RO"   'open-rectangle)
    (define-key ctl-t-map "RW"   'kill-rectangle)
    (define-key ctl-t-map "RY"   'yank-rectangle)
    (define-key ctl-t-map "\C-[" 'backward-paragraph)
    (define-key ctl-t-map "\C-]" 'forward-paragraph)
    (define-key ctl-t-map "\C-a" 'beginning-of-buffer)
    (define-key ctl-t-map "\C-b" 'bury-buffer)
    (define-key ctl-t-map "\C-c" 'ndw/open-calendar)
    (define-key ctl-t-map "\C-e" 'end-of-buffer)
    (define-key ctl-t-map "\C-g" 'keyboard-quit)
    (define-key ctl-t-map "\C-k" 'kill-buffer)
    (define-key ctl-t-map "\C-l" 'ndw/goto-line-with-feedback)
    (define-key ctl-t-map "\C-n" 'next-error)
    (define-key ctl-t-map "\C-r" 'all)
    (define-key ctl-t-map "\C-s" 'sort-lines)
    (define-key ctl-t-map "\C-t" 'wwtime)
    (define-key ctl-t-map "\C-w" 'ispell-word)
    (define-key ctl-t-map "\C-z" 'wwtime-convert)
    (define-key ctl-t-map "b"    'org-brain-visualize)
    (define-key ctl-t-map "c"    'helm-org-contacts)
    (define-key ctl-t-map "d"    'dired)
    (define-key ctl-t-map "e"    'insert-euro)
    (define-key ctl-t-map "f"    'set-fill-prefix)
    (define-key ctl-t-map "l"    'insert-pound)
    (define-key ctl-t-map "m"    'mu4e)
    (define-key ctl-t-map "p"    'ndw-blog:post)
    (define-key ctl-t-map "q"    'xml-quotes-add-mail-signature)
    (define-key ctl-t-map "s"    'ispell-buffer)
    (define-key ctl-t-map "u"    'xmlunicode-character-insert-helm)
    (define-key ctl-t-map "z"    'insert-zero-width-space)
    (define-key ctl-t-map "ox"   'om-to-xml)
    (define-key ctl-t-map "yi"   'yankpad-insert)
    (define-key ctl-t-map "yy"   'yankpad-insert)
    (define-key ctl-t-map "ye"   'yankpad-expand)
    (define-key ctl-t-map "ym"   'yankpad-map)
    (define-key ctl-t-map "ys"   'yankpad-set-category)
  (defvar Control-T-prefix ctl-t-map "Norm's extension prefix.")

Control X bindings

  ;; Changes to Control-X map
  (define-key ctl-x-map "\C-]" 'save-buffer)

Global bindings

;; Changes to global map
(define-key global-map "\C-t" Control-T-prefix)
(define-key global-map "\ej"  'fill-paragraph)
(define-key global-map "\eq"  'query-replace-regexp)
(define-key global-map "\er"  'replace-regexp)
(define-key global-map "#"    'quoted-insert)
(define-key global-map "\eOC" 'forward-word)
(define-key global-map "\eOD" 'backward-word)
(global-set-key "\C-\\" 'call-last-kbd-macro)
(global-set-key "\C-Z"  'undo)
(global-set-key [M-left] 'backward-sexp)
(global-set-key [M-right] 'forward-sexp)

(global-set-key (kbd "M-z") nil)
(global-set-key (kbd "C-x C-z") nil)
(global-set-key (kbd "M-/") nil)

;; Move up/down paragraph
(global-set-key (kbd "M-n") #'forward-paragraph)
(global-set-key (kbd "M-p") #'backward-paragraph)

;; This doesn’t work and I don’t know why
(global-set-key [M-a] 'copy-whole-buffer)

Mode bindings

xmlunicode provides some convenience methods for inserting Unicode characters (in particular for what have traditionally been SGML XML named character entities).

(use-package xmlunicode
  ;; set Unicode data file location. (used by what-cursor-position and describe-char)
  (let ((x "~/.emacs.d/UnicodeData.txt"))
    (when (file-exists-p x)
      (setq describe-char-unicodedata-file x))))

(require 'xmlunicode)
(setq xmlunicode-default-single-quote xmlunicode-rsquo)

I bind them in several modes.

  (defun bind-smart-characters (mode-map)
    "Add bindings for special punctuation handling to MODE-MAP."
    (define-key mode-map "\"" 'xmlunicode-smart-double-quote)
    (define-key mode-map "'" 'xmlunicode-smart-single-quote)
    (define-key mode-map "-" 'xmlunicode-smart-hyphen)
    (define-key mode-map "." 'xmlunicode-smart-period))

  (defun bind-nxml-mode-keys ()
    "Special support for `nxml-mode'."
    (set-language-environment "utf-8")
    (define-key nxml-mode-map (kbd "<C-return>") 'completion-at-point)
    (define-key nxml-mode-map ";" 'xmlunicode-smart-semicolon)
    ; Now that I have C-t C-u, I never use this
    ;(define-key nxml-mode-map [menu-bar unichar]
    ;  (cons "UniChar" xmlunicode-character-menu-map))
    (set-input-method 'xml)
    (bind-smart-characters nxml-mode-map))

  (defun bind-adoc-mode-keys ()
    "Use smart insert characters in `adoc-mode'."
    (bind-smart-characters adoc-mode-map))

  (defun bind-markdown-mode-keys ()
    "Use smart insert characters in `markdown-mode'."
    (bind-smart-characters markdown-mode-map))

  (defun bind-text-mode-keys ()
    "Use smart insert characters in `text-mode'."
    (bind-smart-characters text-mode-map))

  (add-hook 'markdown-mode-hook 'bind-markdown-mode-keys)
  (add-hook 'nxml-mode-hook 'bind-nxml-mode-keys)
  (add-hook 'adoc-mode-hook 'bind-adoc-mode-keys)
  (add-hook 'text-mode-hook 'bind-text-mode-keys)


At the beginning of 2019, I made a decision to seriously embrace Org-mode. Lots of my config is about tinkering with Org-mode. It’s great, except for the parts I really wish were different. :-)

This section sets up Org-mode, it’s prerequisites, and the extensions I’m using.

doct · doct provies “Declarative Org-mode Capture Templates” for Emacs. I find them a little easier to reason about than the standard form.

 '(doct :type git :host github :repo "progfolio/doct"))

Org-mode · I use org-mode for a lot of things. Let’s make sure it’s loaded and configured.

(use-package org-plus-contrib
  :mode (("\\.org$" . org-mode)))
  (require 'ndw-org)

;; Load my LaTeX customizations if we ever load ox-latex.
(with-eval-after-load 'ox-latex
   (require 'ndw-org-latex))

Org-sticky-header · org-sticky-header keeps headings at the top of the buffer. Updates from alhassy.

I actually find this a little distracting and probably won’t continue using it. YMMV.

(use-package org-sticky-header
 :hook (org-mode . org-sticky-header-mode)
  org-sticky-header-full-path 'full
  ;; Child and parent headings are seperated by a /.
  org-sticky-header-outline-path-separator " / "))

org-jira · org-jira claims to bring Jira and OrgMode together.

 '(org-jira :type git :host github :repo "ahungry/org-jira"))

(with-eval-after-load 'org-jira
  ;; I use auth-source to conceal “secrets” from publication.
  ;; I don’t think the MarkLogic JIRA URL is especially secret,
  ;; but the mechanism exists, so…
  (setq jiralib-url
        (let ((found
               (nth 0
                    (auth-source-search :max 1
                                        :host "jiralib"
                                        :require '(:url)
                                        :create nil))))
          (plist-get found :url))))

org-bullets · org-bullets provides UTF-8 bullets for Org-mode.

(use-package org-bullets
  (setq org-bullets-bullet-list '("●" "○" "●" "○" "●" "◉" "○" "◆"))
  (add-hook 'org-mode-hook (lambda () (org-bullets-mode 1))))

org-cliplink · org-cliplink inserts Org-mode links from the clipboard.

(use-package org-cliplink)

org-context · org-context provides contextual agenda and capture for Org-mode.

(use-package org-context)

org-tree-slide · org-tree-slide is a presentation tool for Org-mode based on the visibility of outline trees.

Ultimately, I’m not happy with the way this works and I’ve cobbled together my own set of functions for presentations from Emacs.

(use-package org-tree-slide)

org-journal · org-journal is a simple Org-mode based journaling mode.

 '(org-journal :type git :host github :repo "ndw/org-journal"))

(defun ndw/org-journal-monday (&optional time)
  "Return the TIME cast back to the previous Monday.
If TIME isn’t specified, the (current-time) is assumed.
If today is Monday, return today."
  (let* ((now    (if time time (current-time)))
         (dow    (string-to-number (format-time-string "%u" now)))
         (delta  (if (= dow 1)
                   (* (- dow 1) (* 24 3600)))))
    (seconds-to-time (- (time-to-seconds now) delta))))

(customize-set-variable 'org-journal-file-type 'weekly)
(customize-set-variable 'org-journal-dir "~/Dropbox/Org/Journal/")
(customize-set-variable 'org-journal-file-format "%Y/%m/")
(customize-set-variable 'org-journal-date-format "%A, %d %B %Y")
(customize-set-variable 'org-journal-date-prefix "* ")
(customize-set-variable 'org-journal-time-prefix "** ")
(customize-set-variable 'org-journal-enable-agenda-integration t)

(customize-set-variable 'org-journal-file-header
   (lambda ()                        
     (let ((monday (ndw/org-journal-monday)))
       (insert "#+TITLE: ")
       (insert (format-time-string org-journal-date-format monday))
       (insert "\n#+STARTUP: content\n#+FILETAGS: :Journal:\n\n"))))

(require 'org-journal)

(define-key org-mode-map (kbd "C-c C-j") 'org-journal-new-entry)

org-ref · org-ref is a set of Org-mode modules for citations, cross-references, and bibliographies in Org-mode and useful BibTeX tools to go with it.

I have not figured out org-ref yet.

;; Files to look at when no “╲bibliography{⋯}” is not present in a file.
;; Most useful for non-LaTeX files.
(setq reftex-default-bibliography '("~/etc/bibliography.bib"))
(setq bibtex-completion-bibliography (car reftex-default-bibliography))

(use-package org-ref
  :config (setq org-ref-default-bibliography reftex-default-bibliography)
  :requires (org-bibtex))

;; Quick BibTeX references, sometimes.
(use-package helm-bibtex)
(use-package biblio)

org-mac-iCal · org-mac-iCal allows you to imports events from to the Emacs diary.

 '(org-mac-iCal :type git :host github :repo "ndw/org-mac-iCal"))

(require 'org-mac-iCal)
(setq org-mac-iCal-import-exchange t)
(setq org-mac-iCal-calendar-names
      '("Norman Walsh" "Family" "Home" "Calendar" "US Holidays"))

yankpad · yankpad provides the ability to paste snippets from an Org-mode file.

 '(yankpad :type git :host github :repo "Kungsgeten/yankpad"))

org-ql · org-ql is an Org-mode query language, including search commands and saved views.

;(use-package org-ql)

org-super-agenda · org-super-agenda is an Org-mode enhancement to make the agenda view more organized.

;(use-package org-super-agenda)

org-msg · org-msg is a global minor mode for mixing Org mode and Message mode to compose and reply to emails in an Outlook HTML friendly style.

;(use-package org-msg
;  :requires (emacs-htmlize))

 '(org-msg :type git :host github :repo "jeremy-compostella/org-msg"
           :branch "experimental"
           :requires (emacs-htmlize)))

org-screenshot (org-attach-screenshot) · org-screenshot (org-attach-screenshot) provides screenshots integrated with emacs org mode attachments.

'(org-attach-screenshot :type git :host github :repo "dfeich/org-screenshot"))
(setq org-attach-screenshot-command-line "screencapture -m %f")    

org-brain · org-brain provides Org-mode wiki + concept-mapping.

;; Load org-brain after my personal org settings...
(use-package org-brain
  (setq org-brain-path "~/Dropbox/Org/brain")
  (setq org-id-track-globally t)
  (setq org-id-locations-file "~/.emacs.d/.org-id-locations")

  (push (car (doct '(("Brain" :keys "b"
                      :type plain
                      :function org-brain-goto-end
                      :template "** %i%?"))))

;  (push '("b" "Brain" plain (function org-brain-goto-end)
;          "* %i%?" :empty-lines 1)
;        org-capture-templates)

;  (push (doct '(("Brain" :keys "b"
;         :type plain
;         :function org-brain-goto-end
;         :template "** %i%?")))
;         org-capture-templates)

  (setq org-brain-visualize-default-choices 'all)
  (setq org-brain-title-max-length 12)
  (setq org-brain-include-file-entries nil)
  (setq org-brain-file-entries-use-title nil))

code-library · code-library supports using Org-mode to manage code snippets.

(use-package code-library)


This is the beginning of an attempt to untangle the “unsorted bag of stuff” below.

Let me be clear up front: there are a lot of packages here that I probably tried once or twice and forgot about. The trick is figuring out which ones. And of course, when I try, I’m reminded that they’re kind of cool so I decide to leave them, try them once or twice, and then forget about them again.

Avy · Avy, a nice way to move around text.

(use-package avy
  :defer t
  (("M-z c" . avy-goto-char-timer)
   ("M-z l" . avy-goto-line))
  (avy-timeout-seconds 0.3)
  (avy-style 'pre)
  (avy-lead-face ((t (:background "#51afef" :foreground "#870000" :weight bold)))));

Crux · Crux, a Collection of Ridiculously Useful eXtensions for Emacs.

(use-package crux
  (("C-x 4 t" . crux-transpose-windows)
   ("C-x K" . crux-kill-other-buffers))
  (crux-with-region-or-buffer indent-region)
  (crux-with-region-or-buffer untabify)
  (crux-with-region-or-point-to-eol kill-ring-save)
  (defalias 'rename-file-and-buffer #'crux-rename-file-and-buffer))

Disk usage · disk-usage is a file system analyzer: it offers a tabulated view of file listings sorted by size. Directory sizes are computed recursively.

(use-package disk-usage
  :commands (disk-usage))

Atomic Chrome · Atomic Chrome allows you to use Emacs to edit textarea elements in the browser

(use-package atomic-chrome)

Zel · Zel tracks files by “frecency” of access. Basically a port of z in Emacs.

(use-package zel
  :bind (("C-x C-r" . zel-find-file-frecent))

Diminsh · Dimish removes (certain) minor modes from the mode-line.

(use-package diminish)

Visual line mode tweaks · Visual line mode has been the default since Emacs 23, but I don’t like it.

(setq line-move-visual nil)
(setq visual-line-fringe-indicators '(left-curly-arrow right-curly-arrow))
(use-package visual-fill-column)
(add-hook 'visual-line-mode-hook #'visual-fill-column-mode)
(add-hook 'visual-line-mode-hook #'ndw/adaptive-fill-paragraph-mode)
; On further consideration, no.
;(add-hook 'org-mode-hook 'visual-line-mode) 

   :type git
   :host github
   :repo "luisgerhorst/virtual-auto-fill"))

; On further consideration, no.
;(add-hook 'org-mode-hook #'virtual-auto-fill-mode)

om · om is a functional library for org-mode.

 '(om :type git :host github :repo "ndwarshuis/om.el"))

org-to-xml · org-to-xml is a library to convert Emacs org-mode files to XML.

 '(om-to-xml :type git :host github :repo "ndw/org-to-xml"))

(require 'ndw-om-to-xml)


  • ack is an Emacs interface to ack-like tools.
  • full-ack is an Emacs front-end for ack.

Do I need both of these?

(use-package ack)
(use-package full-ack)

deadgrep · deadgrep provides fast, friendly searching with ripgrep and Emacs.

(use-package deadgrep)

bbdb · bbdb is the Insideous Big Brother Database.

(use-package bbdb
  (defadvice bbdb-read-string (before bbdb-read-string-no-ivy activate) (ivy-mode 0))
  (defadvice bbdb-read-string (after bbdb-read-string-yes-ivy activate) (ivy-mode 1)))

bind-key · The bind-key package is a simple way to manage personal keybindings. It appears to be part of use-package.

(use-package bind-key)

charmap · charmap provides Unicode tables in Emacs. M-x charmap.

(use-package charmap)

color-theme-modern · color-theme-modern is part of replace-colorthemes which replaces color-theme with the Emacs 24 theme framework.

(use-package color-theme-modern)

company · company-mode is a modular in-buffer completion framework. Updates from alhassy.

I’ve turned off global-company-mode it was much too intrusive in non-programming contexts. It may be too intrusive in programming contexts as well.

(use-package company
  (global-company-mode 0)
  (setq ;; Only 2 letters required for completion to activate.
        company-minimum-prefix-length 2

        ;; Search other buffers for completion candidates
        company-dabbrev-other-buffers t
        company-dabbrev-code-other-buffers t

        ;; Allow (lengthy) numbers to be eligible for completion.
        company-complete-number t

        ;; M-⟪num⟫ to select an option according to its number.
        company-show-numbers t

        ;; Edge of the completion list cycles around.
        company-selection-wrap-around t

        ;; Do not downcase completions by default.
        company-dabbrev-downcase nil

        ;; Even if I write something with the ‘wrong’ case,
        ;; provide the ‘correct’ casing.
        company-dabbrev-ignore-case t

        ;; Immediately activate completion.
        company-idle-delay 0))

(add-hook 'prog-mode-hook 'company-mode)


(use-package company-emoji
  :config (add-to-list 'company-backends 'company-emoji))

cperl-mode · cperl-mode is an advanced mode for programming in Perl.

(use-package cperl-mode)

rainbow-mode · rainbow-mode is a minor mode sets background color to strings that match color names, e.g. #0000ff is displayed in white with a blue background.

(use-package rainbow-mode
  (add-hook 'emacs-lisp-mode-hook 'rainbow-mode))

css-mode · css-mode is a mode for editing CSS stylesheets.

(use-package css-mode
  :requires (rainbow-mode)
  (add-hook 'css-mode-hook 'rainbow-mode))

dash · dash is a modern list library for Emacs.

;; Modern list library
(use-package dash)

magit · magit is a git porcelain inside Emacs.

With the default tramp-ssh-controlmaster-options, tramp takes a long time to start and consequently magit takes a long time.

(defvar tramp-ssh-controlmaster-options)
(defvar magit-last-seen-setup-instructions)
(use-package magit
  (setq tramp-ssh-controlmaster-options "")
  (setq magit-last-seen-setup-instructions "1.4.0"))

Docker-related packages

(use-package docker)
(use-package docker-api)
(use-package docker-compose-mode)
(use-package dockerfile-mode)

emojify · emojify displays emojiy in Emacs. Don’t bother loading it unless we’re running in a window system.

(use-package emojify
  :if window-system
  (defun --set-emoji-font (frame)
    "Adjust the font settings of FRAME so Emacs can display emoji properly."
    (if (eq system-type 'darwin)
        ;; For NS/Cocoa
        (set-fontset-font t 'symbol (font-spec :family "Apple Color Emoji") frame 'prepend)
      ;; For Linux
      (set-fontset-font t 'symbol (font-spec :family "Symbola") frame 'prepend)))
  ;; For when Emacs is started in GUI mode:
  (--set-emoji-font nil)
  (add-hook 'after-make-frame-functions '--set-emoji-font)
  (add-hook 'org-mode-hook 'emojify-mode))

exec-path-from-shell · exec-path-from-shell makes Emacs use the $PATH set up by the user's shell

(use-package exec-path-from-shell)

ledger-mode · ledger-mode is a major mode for editing “Ledger CLI” files.

(use-package ledger-mode)

Flycheck-related packages

(use-package flycheck
(use-package flycheck-ledger)
(use-package flycheck-package)

graphviz-dot-mode · graphviz-dot-mode is an Emacs mode for Graphviz DOT files.

(use-package graphviz-dot-mode)

git-gutter · git-gutter shows git changes in the fringe.

(use-package git-gutter
  (setq git-gutter:deleted-sign "▁")
  (setq git-gutter:added-sign "▌")
  (setq git-gutter:modified-sign "▌")
  (global-git-gutter-mode +1))

Hydra · hydra shortens key bindings by allowing you to type the prefix only once. Updates from alhassy.

Use Hydra to navigate through version control changes. I’ve commented out revert and stage because it feels like it would be too easy to do them accidentally.

(use-package hydra)

(defhydra hydra-version-control (global-map "C-x v")
  "Version control"
  ;; Syntax: (extension method description)
  ("n" git-gutter:next-hunk      "Next hunk")
  ("p" git-gutter:previous-hunk  "Previous hunk")
  ("d" git-gutter:popup-hunk     "Show hunk diff")
;  ("r" git-gutter:revert-hunk    "Revert hunk\n")
;  ("c" git-gutter:stage-hunk     "Stage hunk")
  ("s" git-gutter:statistic      "How many added & deleted lines"))

Groovy- and Gradle-related packages

  • groovy-mode is a major mode for editing Groovy files. Also included in the package: REPL integration with run-groovy and Grails project navigation with grails-mode.
(use-package groovy-mode
  (setq groovy-indent-offset 2)
  :mode (("\\.groovy$" . groovy-mode)
         ("\\.gradle$" . groovy-mode)))

projectile · projectile is the Project Interaction Library for Emacs.

(use-package projectile)

restclient · restclient is an HTTP REST client tool for emacs.

(use-package restclient)

helm · helm is an incremental completion and selection narrowing framework for Emacs.

(use-package helm
  (setq helm-split-window-inside-p            t
        helm-move-to-line-cycle-in-source     t
        helm-ff-search-library-in-sexp        t
        helm-scroll-amount                    8
        helm-ff-file-name-history-use-recentf t)
  ;; Remove helm-source-info-cl which I don't seem to have
  (setq helm-info-default-sources
  :bind (("<f1>" .      helm-resume)
         ("<f2>" .      helm-execute-kmacro)
         ("C-," .       helm-calcul-expression)
         ("C-:" .       helm-eval-expression-with-eldoc)
         ("C-c <SPC>" . helm-all-mark-rings)
         ("C-c f" .     helm-recentf)
         ("C-c g" .     helm-gid)
         ("C-c i" .     helm-imenu)
         ("C-c I" .     helm-imenu-in-all-buffers)
         ("C-h C-s" .   helm-occur)
         ("C-h a" .     helm-apropos)
         ("C-h f" .     helm-find)
         ("C-h g" .     helm-google-suggest)
         ("C-h i" .     helm-info-at-point)
         ("C-h r" .     helm-info-emacs)
         ("C-h s" .     helm-swoop)
         ("C-x C-d" .   helm-browse-project)
         ("C-x C-f" .   helm-find-files)
         ("C-x b" .     helm-mini)
         ("C-x r b" .   helm-filtered-bookmarks)
         ("C-x c b" .   helm-chrome-bookmarks)
         ("M-x" .       helm-M-x)
         ("M-y" .       helm-show-kill-ring)
         ("M-g a" .     helm-do-grep-ag)
         ([remap jump-to-register] . helm-register)
         ([remap list-buffers]     . helm-buffers-list)
         ([remap dabbrev-expand]   . helm-dabbrev)
         ([remap find-tag]         . helm-etags-select)
         ([remap xref-find-definitions] . helm-etags-select))
  (require 'ndw-helm)
  (require 'xmlunicode-helm)

helm-projectile · helm-projectile is a Helm UI for projectile.

(use-package helm-projectile
  :requires (helm)
  (setq projectile-enable-caching t)
  (setq projectile-completion-system 'helm)
  ("C-c p" . projectile-command-map)

helm-wordnet · helm-wordnet is a helm interface to WordNet.

(use-package helm-wordnet
  :requires (helm)
  (setq helm-wordnet-wordnet-location "/usr/local/Cellar/wordnet/3.1")
  :bind (:map ctl-t-map
              ("w" . helm-wordnet-suggest)))

restclient-helm · restclient is a tool to manually explore and test HTTP REST webservices. Runs queries from a plain-text query sheet, displays results as a pretty-printed XML, JSON and even images.

(use-package restclient-helm
  :requires (helm))

all-ext · all-ext is an extension-of/replacement for all.

(use-package all-ext
  :requires (helm))

helm-ag · helm-ag is a helm interface to the silver searcher.

(use-package helm-ag
  :requires (helm))

helm-bbdb · helm-bbdb is a helm interface to BBDB.

(use-package helm-bbdb
  :requires (helm))

helm-flycheck · helm-flycheck shows flycheck errors within helm.

(use-package helm-flycheck
  :requires (helm))

helm-swoop · helm-swoop efficiently hops squeezed lines powered by Emacs helm interface .

(use-package helm-swoop
  :requires (helm))

helm-company · helm-company is a helm interface to company-mode.

(use-package helm-company
  :requires (helm))

hyperspace · hyperspace - get there from here.

(use-package hyperspace
  (setq hyperspace-actions
        '(("ddg" . "")
          ("dis" . "")
          ("g"   . "")
          ("gi"  . "")
          ("bb"  . bbdb-search-name)
          ("el"  . (apply-partially #'hyperspace-action->info "(elisp)Top"))
          ("av"  . apropos-variable)
          ("ac"  . apropos-command)
          ("af"  . (lambda (query) (apropos-command query t)))

          ("gh"   . "")
          ("wiki" . "")
          ("gm"   . "")))
  :bind (:map hyperspace-minor-mode-map
              ("C-c s" . hyperspace))

multiple-cursors · multiple-cursors gives Emacs…multiple, parallel cursors

(use-package multiple-cursors
  :bind (("C-S-c C-S-c" . mc/edit-lines)
         ("C->" . mc/mark-next-like-this)
         ("C-<" . mc/mark-previous-like-this)
         ("C-c C-<" . mc/mark-all-like-this)))

markdown-related packages · Mostly I work in Org-mode these days, so I don’t use these as often as I once did.

  • markdown-mode is a major mode for editing Markdown files.
  • markdown-mode+ adds additional functions for Emacs markdown-mode. Default support for [pandoc]. Much of the functionality is tailored to OS X, with the end goal to extend markdown-mode in useful ways for all platforms.
  • markdown-preview-mode is a minor mode to preview Markdown files as you save.
(use-package markdown-mode)
(use-package markdown-mode+)
(use-package markdown-preview-mode)

markup-faces · markup-faces is a collection of faces for markup language modes.

(use-package markup-faces)

rainbow-delimiters · rainbow-delimiters marks nested delimeters (parenthesis, brackets, etc.) in different colors.

(use-package rainbow-delimiters
  (add-hook 'prog-mode-hook #'rainbow-delimiters-mode))

recentf · recentf maintains a list of recently edited files.

(use-package recentf
  (setq recentf-max-menu-items 100)
  (recentf-mode 1))

rnc-mode · rnc-mode is a major mode for editing RELAX NG compact syntax files.

(use-package rnc-mode
  (setq rnc-indent-level 3))

s · s is the long lost Emacs string manipulation library.

(use-package s)

sass-mode · sass-mode is a major mode for editing SASS files.

(use-package sass-mode)

Javascript and JSON-related packages

(use-package indium)
(use-package js2-mode
  (setq js-indent-level 2)
  :mode (("\\.js" . js2-mode)
         ("\\.sjs" . js2-mode))
  (add-hook 'js2-mode-hook #'js2-imenu-extras-mode)) ;; Better imenu
(use-package xref-js2)
(use-package json-mode
  :mode (("\\.json\\'" . json-mode)
         ("\\manifest.webapp\\'" . json-mode )
         ("\\.tern-project\\'" . json-mode)))

Typescript-related packages

  • typescript-mode is a major mode for editing typescript files.
  • tide is the TypeScript Interactive Development Environment for Emacs.

Do I need both of these?

(use-package typescript-mode)
(use-package tide)

web · web is a useful HTTP client in Emacs Lisp

(use-package web)

yaml-mode · yaml-mode is a major mode for editing YAML files.

(use-package yaml-mode)

htmlize · htmlize converts a buffer of text and decorations into HTML.

(use-package htmlize)

calfw-related packages · calfw is a calendar framework for Emacs.

(use-package calfw
  (setq cfw:fchar-junction ?╋
        cfw:fchar-vertical-line ?┃
        cfw:fchar-horizontal-line ?━
        cfw:fchar-left-junction ?┣
        cfw:fchar-right-junction ?┫
        cfw:fchar-top-junction ?┯
        cfw:fchar-top-left-corner ?┏
        cfw:fchar-top-right-corner ?┓))
(use-package calfw-org)
(use-package calfw-ical)
(use-package calfw-cal)

;;; My calfw customizations
(defun ndw/open-calendar ()
  "CFW config for my calendars."
    (cfw:org-create-source "Green")
    ;(cfw:cal-create-source "Orange")
    (cfw:ical-create-source "Home" "~/Documents/Home.ics" "Gray")
    (cfw:ical-create-source "Family" "~/Documents/Family.ics" "Orange")

wwtime · wwtime is an emacs library to insert time-of-day with appropriate world-wide localization.

(use-package wwtime
  ;(setq wwtime-display '("EST" "PST" "GMT" "CET" "JST" "India"))
  ;(setq wwtime-display '("EDT" "PDT" "GMT" "BST" "CEST" "JST" "India"))
  ;(setq wwtime-display '("CDT" "PDT" "GMT" "India"))
  (setq wwtime-display '("CST" "PST" "GMT" "India"))
  ;(setq wwtime-display '("CDT" "PDT" "GMT" "India"))
  (setq wwtime-ampm '("EST" "CST" "MST" "PST" "AKST"
                      "EDT" "CDT" "MDT" "PDT" "AKDT"
  (setq wwtime-time-zones (append wwtime-time-zones '(("India" +5.5 "India")))))

xml-quotes · xml-quotes provides macros for generating quotations from an XML document.

(defvar signature-quote-number t)
(use-package xml-quotes
  (setq xml-quotes-message-signature-file "~/.signatures/default")
  (setq xml-quotes-closing-name "norm")
  (setq xml-quotes-closing-text-alist '(("#default" "Be seeing you," "Cheers,")))
  ;; These are “gnus” groups.
  (setq xml-quotes-group-no-closing '("docbook.admin"
  (setq xml-quotes-default-signature-function 'xml-quotes-gnus-default-signature)
  (defun set-signature-quote (&optional prefixarg)
    (interactive "P")
    (if prefixarg
        (if (numberp prefixarg)
            (setq signature-quote-number prefixarg)
          (setq signature-quote-number t)))
    (if (numberp signature-quote-number)
        (message "Next signature quote is #%d" signature-quote-number)
      (message "Next signature quote is random"))))

synosaurus · synosaurus is an extensible thesuarus mode for Emacs.

(use-package synosaurus)

powerthesaurus · powerthesaurus is a PowerThesaurus integration for Emacs.

(use-package powerthesaurus)

websocket · websocket is an Emacs Lisp implementation of websockets.

(use-package websocket)

writegood-mode · writegood-mode is a minor mode to aid in finding common writing problems.

I’m not wholly satisfied with this mode, to be honest. In particular, it tends (IMHO) to be overly aggressive about passive voice. Or maybe my technical writing just uses too much passive voice.

(use-package writegood-mode
  (add-hook 'nxml-mode-hook 'writegood-mode)
  (add-hook 'org-mode-hook 'writegood-mode))

beacon · beacon highlights the line the cursor is on whenever the window scrolls.

(use-package beacon
  (beacon-mode 1))

wsd-mode · wsd-mode is a major-mode for Emacs and

(use-package wsd-mode)

jdee · jdee is an add-on software package that turns Emacs into a comprehensive system for creating, editing, debugging, and documenting Java applications.

I confess, I’ve never really been able to set this up well enough. I mostly use IntelliJ for Java and Scala.

(use-package jdee)

git-timemachine · git-timemachine allows you to step through historic versions of git controlled files.

(use-package git-timemachine)

tldr · tldr is a tldr client for Emacs.

(use-package tldr)

python-mode · python-mode is a major mode for editing Python.

(use-package python-mode
  :requires (flycheck-mode)
  (setq py-shell-name "python3")
  (setq flycheck-python-pylint-executable "/Users/ndw/.pyenv/shims/pylint"))

plantuml-mode · plantuml-mode is a major mode for editing PlantUML diagrams.

(use-package plantuml-mode
  (setq plantuml-jar-path "/Users/ndw/java/plantuml.1.2019.9.jar"))

elpy · elpy is the Emacs Python Development Environment.

(use-package elpy

company-restclient · company-restclient is a company-mode completion back-end for restclient.

(use-package company-restclient)

emacs-request · emacs-request is a library that simplifies HTTP interactions from Emacs.

 '(emacs-request :type git :host github :repo "tkf/emacs-request"))

ob-restclient · ob-restclient is an Org-mode extension for restclient.

(use-package ob-restclient)

ob-ml-marklogic · ob-ml-marklogic provides org-babel integration with MarkLogic XQuery and (server side) JavaScript code blocks

 '(ob-ml-marklogic :type git :host github :repo "ndw/ob-ml-marklogic"))
(require 'ndw-ob-ml-marklogic)

xproc-mode · xproc-mode provides a trivial variant of nxml-mode that supports org-babel evaluation.

 '(xproc-mode :type git :host github :repo "ndw/xproc-mode"))
(require 'xproc-mode)
(setq xproc-processor "/Users/ndw/bin/meerschaum")

minions · minions is a minor-mode menu for the mode-line.

(use-package minions

annotate · annotate provides a minor mode which can add annotations to arbitrary files without changing the files themselves. This is very useful for code reviews. When annotate-mode is active, C-c C-a will create, edit, or delete annotations.

I haven’t established the habit of using it.

(use-package annotate)

sparql-mode · sparql-mode is a major-mode for editing SPARQL files.

(use-package sparql-mode)

undo-propose · undo-propose allows you to navigate the emacs undo history by staging undo's in a temporary buffer.

(use-package undo-propose)

dimmer · dimmer interatively highlights which buffer is active by dimming the others.

(use-package dimmer

dashboard · dashboard presents an extensible Emacs dashboard.

(use-package dashboard

company-marklogic · company-marklogic is a company backend for MarkLogic functions.

(require 'company)
 '(company-marklogic :type git :host github
                     :repo "fgeorges/company-marklogic"
                     :files (:defaults "src/*.el")))

(require 'company-marklogic)
(add-to-list 'company-backends 'company-marklogic-sjs)
(add-to-list 'company-backends 'company-marklogic-xqy)
(add-hook 'xquery-mode-hook 'company-mode)

helm-org-contacts · helm-org-contacts is a helm source for Org contacts.

 '(helm-org-contacts :type git :host github :repo "tmalsburg/helm-org-contacts"))

(require 'org-contacts)

(setq org-contacts-email-property "EMAIL")
(setq org-contacts-tel-property "PHONE")
(setq org-contacts-address-property "ADDRESS")
(setq org-contacts-birthday-property "BIRTHDAY")
(setq org-contacts-note-property "NOTE")
(setq org-contacts-alias-property "ALIAS")
(setq org-contacts-ignore-property "IGNORE")

(setq org-contacts-files

xquery-mode · xquery-mode is a major mode for editing XQuery files.

 '(xquery-mode :type git :host github :repo "ndw/xquery-mode"))

poet-theme · poet-theme is an emacs theme that's well suited for modes using variable pitch: particularly org-mode and markdown-mode.

(use-package poet-theme)
(require 'ndw-theme)
(require 'ndw-fonts)

emacs-gfs · emacs-gfs is a “global face scaling” library.

 '(emacs-gfs :type git :host github :repo "ndw/emacs-gfs"))

(require 'emacs-gfs)
;; Adjust face sizes with +/-
(global-set-key (kbd "C-+") #'gfs/magnify-faces)
(global-set-key (kbd "C--") #'gfs/shrink-faces)

emacs-htmlize · emacs-htmlize converts buffer text and decorations into HTML.

 '(emacs-htmlize :type git :host github :repo "hniksic/emacs-htmlize"))

perspective · perspective lets you group sets of buffers and navigate between them distinctly from the whole buffer list. From alhassy.

Also something I haven’t really incorporated into daily use.

(use-package perspective
  :config ;; Activate it.
          ;; In the modeline, tell me which workspace I'm in.

flyspell · From alhassy.

(use-package flyspell
  :hook ((prog-mode . flyspell-prog-mode)
         (org-mode . flyspell-mode)
         (text-mode . flyspell-mode)))

(setq ispell-list-command "list")
(setq ispell-program-name "/usr/local/bin/aspell")
(setq ispell-dictionary "en_US") ;; set the default dictionary

(setq  ispell-extra-args '("--sug-mode=ultra"))

;; This run-together option causes a whole bunch of
;; misspellings to be treated as correct. WTF?
;;                            "--run-together"
;;                            "--run-together-limit=5"
;;                            "--run-together-min=2"))

(eval-after-load "flyspell"
  ' (progn
     (define-key flyspell-mouse-map [down-mouse-3] #'flyspell-correct-word)
     (define-key flyspell-mouse-map [mouse-3] #'undefined)))

;; (custom-set-faces '(flyspell-incorrect ((t (:inverse-video t)))))

(setq ispell-silently-savep t)

(add-hook          'c-mode-hook 'flyspell-prog-mode)
(add-hook 'emacs-lisp-mode-hook 'flyspell-prog-mode)

(use-package synosaurus
  :diminish synosaurus-mode
  :init    (synosaurus-mode)
  :config  (setq synosaurus-choose-method 'popup) ;; 'ido is default.
           (global-set-key (kbd "M-#") 'synosaurus-choose-and-replace))

(use-package wordnut
 :bind ("M-!" . wordnut-lookup-current-word))

goto-chg · goto-chg moves through a buffer based on the location of the last changes.

; M-. can conflict with etags tag search. But C-. can get overwritten
; by flyspell-auto-correct-word. And goto-last-change needs a fast key.
(use-package goto-chg
  :bind (("C-." . goto-last-change)   
         ("M-." . goto-last-change-reverse)))

(defhydra hydra-edits (global-map "C-c e")
  ("p" goto-last-change "Goto nᵗʰ last change")
  ("n" goto-last-change-reverse "Goto more recent change"))

mu4e · mu4e is installed with brew. It’s an alternative to gnus. I’m playing with it.

;; commented out here because it requires additional config
;;(add-to-list 'load-path "/usr/local/ndw/share/emacs/site-lisp/mu4e")
;;(require 'ndw-mu4e)

Unsorted bag of stuff

More work required here!

;; Make a custom -primary command line argument
 '("primary" . (lambda (&rest ignore)

;; Make a custom -resize command line argument. I don’t
;; find this very effective on the Mac so I’ve mostly stopped
;; using it.
 '("resize" . (lambda (&rest ignore)
                (add-to-list 'default-frame-alist '(top . 0))
                (add-to-list 'default-frame-alist '(left . 0))
                (add-to-list 'default-frame-alist '(width . 108))
                (add-to-list 'default-frame-alist ' (height . 43)))))

;; Autoloads
(setq auto-mode-alist
       (list (cons "\\.txt$" 'text-mode)
             (cons "\\.tex$" 'latex-mode)
             (cons "\\.sli$" 'latex-mode)
             (cons "\\.bib$" 'bibtex-mode)

             (cons "\\.dss?s?l$" 'dsssl-mode)
             (cons "\\.css$" 'css-mode)

             (cons "\\.pl$" 'perl-mode)
             (cons "\\.cls$" 'perl-mode)
             (cons "\\.sup$" 'perl-mode)

             (cons "\\.py$" 'python-mode)

             (cons "\\.rb$" 'ruby-mode)

             (cons "\\.3l$" 'nroff-mode)

             (cons "\\.ttl$" 'ttl-mode)
             (cons "\\.n3$" 'ttl-mode)

             (cons "\\.ts$" 'ng2-ts-mode)

             (cons "\\.rdf$" 'nxml-mode)
             (cons "\\.rnc$" 'rnc-mode)
             (cons "\\.rng$" 'nxml-mode)
             (cons "\\.xpd$" 'nxml-mode)
             (cons "\\.xml$" 'nxml-mode)
             (cons "\\.xpl$" 'nxml-mode)
             (cons "\\.xsd$" 'nxml-mode)
             (cons "\\.xqy$" 'xquery-mode)
             (cons "\\.html$" 'nxml-mode)
             (cons "\\.htm$" 'nxml-mode)
             (cons "\\.xsl$" 'nxml-mode)

(setq magic-mode-alist '(("<\\?xml " . nxml-mode)
                         ("%![^V]" . ps-mode)
                         ("# xmcd " . conf-unix-mode)))

;; for viewing lines matching regexps
(autoload 'all "all" nil t)

;; for RFCs
(autoload 'rfc "rfc" nil t)

;; Various modes
(autoload 'tar-mode "tar-mode.elc" "Tar archive mode." t)
(autoload 'ruby-mode "ruby-mode.elc" "Ruby mode" t)
(autoload 'xquery-mode "xquery-mode.elc" "XQuery mode" t)
(autoload 'python-mode "python-mode" "Mode for editing Python programs" t)
(autoload 'n3-mode "n3-mode" "Mode for editing N3" t)

;; Setup encryption
(setq password-cache-expiry (* 60 60 4)) ; 4 hours

;; Make buffer names unique
(require 'uniquify)

(setq c-mode-hook
      '(lambda ()
         (setq case-fold-search nil)))

(setq term-setup-hook
      '(lambda ()
           (defvar CSI-map nil)
           (and CSI-map (enable-arrow-keys))
           (define-key global-map "\eOR" 'start-xon-isearch))))


 '(ttl-mode :type git :host github
            :repo "jeeger/ttl-mode"))

 '(n3-mode :type git :host github
           :repo "kurtjx/n3-mode-for-emacs"))

(use-package poet-theme)

;; Now that all the packages are setup; do some more config

(require 'ndw-nxml)

;;; init.el ends here

Tangled files


;;; early-init.el --- -*- lexical-binding: t -*-
;; N.B. This file is generated automatically from literate sources.
;; Do not edit this file directly. Your changes will be lost
;; the next time the sources are tangled.
;; Filename: early-init.el
;; Description: Early initialization
;; Author: Norman Walsh
;; Original Author: Mingde (Matthew) Zeng
;; URL:
;; Compatibility: emacs-version >= 27
;;; Commentary:
;; Emacs 27 introduces early-init.el, which is run before init.el,
;; before package and UI initialization happens.
;;; Code:






(provide 'early-init)

;;; early-init.el ends here



;; This is Norm’s Emacs init.el file.

;; N.B. This file is generated automatically from literate sources.
;; Do not edit this file directly. Your changes will be lost
;; the next time the sources are tangled.

;; package.el and to use use-package in a more idiomatic fashion.

;;; Commentary:

;;; Code:









































































































































My configuration files.

Volume 4, Issue 15; 29 Feb 2020

My configuration files.

After I used Org-mode to give a recent presentation, several folks asked

Marginal note:If that’s why you’re here, you may want to focus on the poet-theme and Org-related sections and ignore the rest.

to see my Emacs configuration. I’ve learned a lot from the configurations published by others, so I packaged it up and published it:

It goes without saying, but I’m saying it anyay, that this is a configuration that works for me. I’m not proposing that it should work for you exactly as you find it there. Steal the bits you like. Delete the bits you don’t like. There’s nothing precious about it. Make your configuration work for you!

A few notes:

  1. I’ve named the directory emacs.d (without the leading “.”) to make it a little less unusual in the repository. You should move or copy it to ~/.emacs.d if you want to try it out.
  2. The init.el and early-init.el files are actually generated by tangling the file. If you want to preserve the “literate programming” nature of the configuration, you should change and then use C-c C-v t to regenerate the Emacs lisp files.
  3. I use the straight package manager. If your system is configured with git and some facility (such as ssh-add) for caching credentials, the whole system should bootstrap itself entirely. Note, however, that it requires an internet connection and it takes a good long while the first time you start it!
  4. There are a few things here that aren’t exactly like my config. I’ve left out some bits that rely on private repositories, on configuration for my employer’s systems, and mu4e which requires additional configuration.
  5. I also changed the configuration of Org agenda files so that they don’t rely on ~/Dropbox. You will want to figure out how you prefer to organize your agenda files. (I put mine in Dropbox so that they’re accessible to beorg on my phone).
  6. I explore and experiment with Emacs packages all the time; there’s probably a whole bunch of stuff in there that I’ve used twice and never used again. That doesn’t worry me. It took a couple of decades before I got hooked on Org.

Questions, comments, etc. most welcome.

Adaptive fill mode

To fill or not to fill, that is the question. Well. No. It’s to fill or unfill, really. Which is just…oh, nevermind.

Volume 4, Issue 11; 31 Jan 2020

To fill or not to fill, that is the question. Well. No. It’s to fill or unfill, really. Which is just…oh, nevermind.

I’ve been using Emacs for a quarter of a century or something. Lots of my habits were inculcated in the previous millennium. As a consequence, I don’t tend to treat text the way users of word processors do. I don’t just keep typing and let the editor wrap the lines where it wants. I behave a lot more like I’m using a manual typewriter.

Marginal note:I realize with some emotion, let’s call it amusement, that many readers will never have encountered a typewriter. A typewriter was a mechanical device that allowed you to make ink marks on sheets of pressed, dead trees by banging on big mechanical levers with your fingers. It worked (mostly), but you had to physically “return the carriage” at the end of each line.

On the web and in XML (and in TeX, and in most rational systems), this is fine. Those environments have markup for paragraphs and they know that text in those paragraphs can be reformatted (or “filled”) to fit the display width.

But recently, I’ve encountered a few places where my “manually filled” paragraphs produce odd looking results. One place is in other people’s email readers, where I guess the expectation is that paragraphs will be one long unbroken line that’s refilled. Another is in comments on web pages.

I’m going to keep using Emacs until you pry it out of my cold, dead fingers, but that doesn’t mean I can’t adapt and blend in. So I wondered if I could learn to stop filling paragraphs by hand.

Baby steps: let’s see if I can learn to use it in Org-mode where what I’m typing is usually, mostly prose. It’s also how I take minutes, which I send to people in email, so if it looks nicer in their email readers, that’s a win for me.

First step, enable visual-line-mode. Ugh. That works, but I have a wide Emacs window and I don’t really want lines that are the whole width of the screen. Enter visual-fill-column. So far so good.

My next problem is that I reflexively hit M-j to refill paragraphs as I’m typing. Sure, I’m used to hitting return where I want line breaks in my paragraphs, but as I edit them and rephrase things, the lines get ragged. M-j calls fill-paragraph which reformats all the lines so that they’re “filled” to the right width.

Unfortunately, that inserts hard newlines and I’m back to where I started. Well, I thought, I can fix that:

(defun ndw/fill-paragraph (arg &optional region)
  "Fill the current paragraph with sensitivity to `visual-line-mode`.
In `visual-line-mode`, 'fill' the paragraph by effectively
unfilling it. That let's `visual-line-mode` do the wrapping. In the
absence of `visual-line-mode`, just fill the paragraph as usual. If
ARG is true, do the reverse. If REGION is provided, do the
filling to the paragraphs in the region."
  ;; Parts gleefully stolen from
  (interactive "P")
  (let ((unfill (and visual-line-mode (not arg))))
    (if unfill
        (let ((fill-column (point-max))
              ;; This would override `fill-column' if it's an integer.
              (emacs-lisp-docstring-fill-column t))
          (fill-paragraph nil region))
      (fill-paragraph nil region))))

Sweet. With that bound to M-j, paragraphs are “filled” or “unfilled” according to whether or not the current buffer has visual-line-mode enabled.

Except in Org-mode, M-j is bound to org-fill-paragraph and that’s…complicated.

Rolling up my sleeves and digging into the source, I eventually work out what big function I’d have to redefine to insert my adaptive code into Org. And I just don’t wanna. It’s too complicated and I don’t like redefining big hunks of code from other packages.

That’s when I remembered “advice”. I’ve never used it, but I wondered if it could come to the rescue here. Advice let’s you wrap your own code around every call to a particular function. You can “advise” the behavior of the function without redefining every place where it’s used.

If you dig your way through the Org source code, you will eventually find the business end of M-j (for paragraphs):

		 (dolist (c (delq end cuts))
		   (fill-region-as-paragraph c end justify)
		   (setq end c))))

Right. So. In theory, I should be able to use advice to insinuate myself into the call to fill-region-as-paragraph.

(defun ndw/adaptive-fill-paragraph (orig-fun &rest args)
  (if visual-line-mode
      (let ((fill-column (point-max))
            ;; This would override `fill-column' if it's an integer.
            (emacs-lisp-docstring-fill-column t))
        (apply orig-fun args))
    (apply orig-fun args)))

(advice-add 'fill-region-as-paragraph :around #'ndw/adaptive-fill-paragraph)

And it works! Magical!

Except there’s no way to not do it. There’s no way to not “unfill” in visual-line-mode. It’s not like you’ve bound a key to ndw/fill-paragraph and you can simply run M-x fill-paragraph instead. No, fill-paragraph eventually calls fill-region-as-paragraph (apparently) and you’ve put advice around that so calling fill-paragraph has exactly the same effect as M-j.

Enter a new minor mode:

(define-minor-mode ndw/adaptive-fill-paragraph-mode
  "Toggle adaptive fill paragraph mode.
When enabled, and when `visual-line-mode` is enabled,
`ndw/adaptive-fill-paragraph` fills a paragraph by 'unfilling'
  :init-value nil)

This does nothing except control ndw/adaptive-fill-paragraph, which is redefined thus:

(defun ndw/adaptive-fill-paragraph (orig-fun &rest args)
  (if (and visual-line-mode ndw/adaptive-fill-paragraph-mode)
      (let ((fill-column (point-max))
            ;; This would override `fill-column' if it's an integer.
            (emacs-lisp-docstring-fill-column t))
        (apply orig-fun args))
    (apply orig-fun args)))

(I suppose I could have left this part out and simply toggled visual-line-mode instead, but nevermind.)

All that’s left is to enable these things in Org-mode and see how I get on.

(setq visual-line-fringe-indicators '(left-curly-arrow right-curly-arrow))
(use-package visual-fill-column)
(add-hook 'visual-line-mode-hook #'visual-fill-column-mode)
(add-hook 'visual-line-mode-hook #'ndw/adaptive-fill-paragraph-mode)
(add-hook 'org-mode-hook 'visual-line-mode)

I’d be a little surprised if I was the first person to do something like this, but I enjoyed the exercise.

Converting nnml to Maildir

Converting “nnml” formatted messages to Maildir formatted messages. Crudely.

Volume 4, Issue 8; 25 Jan 2020

Converting “nnml” formatted messages to Maildir formatted messages. Crudely.

I’m feeling stressed and frustrated, so naturally my brain is inventing all kinds of displacement activities to keep me distracted. I’ll have to force myself to confront the real issues eventually, but first, let me do just this one little thing…

I wanted to play with a bit of software that needs a corpus of email messages in Maildir format. Trouble is, all my email is stored in nnml format, an eccentricity of Gnus.

No problem, I figured, I’ll just convert them. Except, I couldn’t find any tools that would do that. I glanced at the Maildir spec and decided that writing one was not a simple enough task to be a displacement activity!

I’d resigned myself to tinkering with a little trickle of messages that I could get over a few days, (or maybe downloading all of my messages from GMail) when I happened randomly across the Python reference page for mailbox.

Could I just brute force it in Python, I wondered?

Yes, yes, I could.

import os
import sys
import re
from mailbox import Maildir
from email.parser import BytesParser

ROOT = "Mail"

total = 0
parser = BytesParser()
for root, dirs, files in os.walk(ROOT):
    if root != "Mail":
        folder = root[5:].replace("/", ".")

        maildir = None
        count = 0
        for name in files:
            if re.match("^\d+$", name):
                if not maildir:
                    os.mkdir("/tmp/x/Maildir/%s" % folder)
                    os.mkdir("/tmp/x/Maildir/%s/cur" % folder)
                    os.mkdir("/tmp/x/Maildir/%s/new" % folder)
                    os.mkdir("/tmp/x/Maildir/%s/tmp" % folder)
                    maildir = Maildir("/tmp/x/Maildir/%s" % folder)
                fn = os.path.join(root, name)
                if count != 0 and count % 100 == 0:
                    print("Added %d to %s" % (count, folder))
                message = None
                with open(fn, "rb") as mail:
                    message = parser.parse(mail)
                count = count + 1
        print("Added %d to %s" % (count, folder))
        total = total + count

print("Total messages: %d" % total)

The nnml format is just directories full of messages in individual files named “1”, “2”, “3”, etc. There may be a few other files in there so I ignore any with names that don’t match ^\d+$. I’m making no effort to preserve the status of messages (read, unread, marked, etc.).

In nnml, directories can be nested and I don’t know if Maildir supports that so I turn Mail/one/two/three into Maildir/one.two.three. It took all of about ten minutes to bang together and a good bit longer to process 150k+ messages. You’ll have noticed that I’m shoving them all into /tmp/x/Maildir, well out of harm’s way so that I can sanity check them before dropping them into ~/Maildir.

I had no idea if that would be sufficient, but it appears to work, so: win!

It’s crude and it’s ugly but if you, random reader from the future, happened to type “convert nnml into Maildir” into your favorite search engine and it got you here, “you’re welcome.” I hope it works for you!

Org to XML

Of course I want XML out! I always want XML out.

Volume 4, Issue 5; 20 Jan 2020

Of course I want XML out! I always want XML out.

A while back, I mentioned in passing that I had worked out how to convert my Org files into XML. One of the reasons I only mentioned it in passing was that I didn’t think I had done it very well.

There were two issues: first, my Emacs lisp coding skills are rusty. There’s a lot of richness in the Org mode data structures and I wasn’t very efficiently or effectively getting information out of them. Second, I realized that what I wanted was an API that made the data structures more regular and easier to access. I wasn’t really in a position to write that because see the first issue.

Time passed and Nate Dwarshuis published a fabulous functional API for org-mode: om.el. That totally resolved the second issue.

I spent a few hours on the plane yesterday rewriting my org-to-xml code on top of that library. The new library, om-to-xml, supercedes org-to-xml, though I’ve left that in the package for the time being. My Emacs lisp may still be a bit amateurish, but it’s getting better. Leveraging Nate’s work also made everything easier.

The goal, as before, is not to have an XML export of the Org mode file (like the HTML or TeX exports), but rather to have an XML version of the Org mode data structures. What this means is that todo items, deadlines, links, macros, property drawers, etc. are all preserved in the output. That makes the Org mode artifact (more) amenable to downstream processing with XML tools.

Some features:

  • Better handling of subscripts and superscripts.
  • Support for macros and other Org mode features I had previously missed.
  • Better handling of source blocks.
  • A small amount of extensibility.

The extensibility hooks consist of some simple configurations (ignored properties, support for insignificant newlines in the XML, etc.) and the ability to write your own function for handling specific node types.

For example, I use a couple of different kinds of blocks in my weblog posts. One of them is for marginal

Marginal note:Like this one.


Like this one.

To Org mode, that’s just a “special block” with a particular type. It’s more convenient for me if it becomes its own element. To achieve that, I add my own handler:

(add-to-list 'om-to-xml-element-handlers
             '(special-block . ndw/om-special-block-to-xml))

That handler deals with a couple of different special block types, outputting my own custom XML markup.

It may still be a niche project, but it’s a better implementation of a niche project!

Emacs “Global Face Scaling”

An Emacs customization to scale all of the faces at the same time.

Volume 4, Issue 3; 08 Jan 2020

An Emacs customization to scale all of the faces at the same time.

To be honest, I don’t need to scale fonts faces

Marginal note:Emacs calls the collection of attributes (size, shape, color, font, etc.) associated with a bit of displayed text a “face”. I’m trying to use that term consistently in this post, even though the font height is the only thing I’m actually changing.

in Emacs very often. I have very occasionally projected an Emacs buffer during a conference or presentation and wished to change the face height.

I usually got by with Options→Set Default Font. Eventually, I learned about text-scale mode and bound the predictable keys, C-+ and C--​, to increase and decrease the face height in steps. As far as I can tell, text-scale mode only changes the height of the default face. Back in the day, when most of my emacs buffers were a single, fixed-width face, that would have been fine.

Today, many of my important buffers have several faces at different sizes. In fact, it was a brief investigation of org-tree-slide over the weekend that started me down this road. I’m intrigued by the idea of just using Emacs to make presentations.

It seems like the limitations of the built-in text scaling would be problematic to lots of people, so I did a little web searching for a workaround. I found a stack overflow question that seemed relevant, and indeed it works for just a face or two, but I wasn’t pleased with the results of my efforts to apply it to an arbitrarily large set of faces.

It’s quite possible that someone else has done a better job of this, but I didn’t find it before solving the problem myself

Marginal note:No, it’s not a displacement activity. Shut up.

seemed more amusing that continued searching.

Thus, emacs-gfs was born.

Here’s how it works. First, find all the faces that have a defined height (an integer). Faces can inherit from one another, so they don’t all have a defined height, and the height can be a floating point scaling factor or a function. I’m assuming if I scale the ones that have a fixed height, inheritance, scaling, and functions will “do the right thing”.

One complication: I don’t necessarily want to scale the mode line or the minibuffer. To avoid scaling these, I added a list of faces that are ignored for scaling.

One more complication: to avoid scaling these faces, I have to fix their height so that, for example, scaling a face they inherit from doesn’t cause them to be scaled. I attempt to do this by following the inheritance chain until I find a height

Marginal note:Known limitation: if any of them have a height that’s defined with a scaling factor or a function, that’s going to get overwritten by the default height.

and the fixing them to have that height. Some faces bottom out without having any defined height at all. (I have no idea how Emacs decides what height the should be.) If that happens, I assign them a default height.

Right. Now there’s a list of faces to scale. If you run gfs/magnify-faces, they’re all scaled up by gfs/magnify-factor. If you run gfs/shrink-faces, they’re all scaled down by that factor. Bind those functions to C-+ and C-- and you can easily scale all the faces in the display up or down.

There are minimum and maximum scaling sizes. I gather that if you scale a face down to less than 1 pixel in height, or larger than 0xffff pixels, bad will happen. If you scale past either of these limits (gfs/face-min-height and gfs/face-max-height), the relative height of faces will be lost.

Customizing org-mode LaTeX output

A summary of some Emacs customizations to make org-mode PDF output a little nicer.

Volume 4, Issue 2; 05 Jan 2020 (updated 07 Jan 2023)

A summary of some Emacs customizations to make org-mode PDF output a little nicer.

[Note: This posting was updated following a thread of useful comments on Twitter.]

Writing in Org-mode strikes a nice balance (for me) between the simplicity of plain text and the richness possible with XML (and it integrates into Emacs so nicely!). I can convert to XML if I need to, but lots of times, nearly-plain-text is good enough.

If you’re asking colleagues to review specifications, however, you might get a little pushback on Org mode files. Nice as they are, if you aren’t used to the Org-mode conventions, they can be distracting. You’re not doing yourself, or your reviewers, any favors by distracting them from your content with the presentation.

Publishing Org documents on the web is easy enough: export them as HTML, update the styles if you wish, and you’re done. Publishing them in a more paginated form is also easy enough: export to PDF by way of LaTeX. I’ve actually come to prefer PDF, sad to say as “a web guy”, for review because I have good tools for reading and annotating them on my tablet and because everyone is happy to review them.

Out-of-the-box, what you get from Org-to-PDF export is a default LaTeX article. In an academic context, where I last encountered it, that was fine. In a non-academic context, I don’t think it works as well.

Default output

(Actually, with the table of contents turned off, that doesn’t look too bad.)

I wrote a book on TeX in the last millenium, but I haven’t thought about it much in twenty years or so. I reached out to Peter Flynn for some advice and in short order had a LaTeX document that I liked much better than the default. It’s all in the preamble.

This is the preamble Peter proposed:

\setmainfont{Linux Libertine O}
\setmonofont[Scale=MatchLowercase]{Luxi Mono}

And this post is an attempt to write the post I wish I’d found when I was searching for how to update my Org-mode configuration to produce the preamble I wanted!

Here are the customizations that work for me:

First, because I’m using UTF-8, switch the processor and configure the process command:

(setq org-latex-compiler "xelatex")
(setq org-latex-pdf-process
      (list (concat "latexmk -"
                    " -recorder -synctex=1 -bibtex-cond %b")))

Then, configure Org to use lstlisting for source environments:

(setq org-latex-listings t)

Next, update org-latex-default-packages-alist to remove inputenc and fontenc because fontspec in my premable takes care of those things:

(setq org-latex-default-packages-alist
      '(("" "graphicx" t)
        ("" "grffile" t)
        ("" "longtable" nil)
        ("" "wrapfig" nil)
        ("" "rotating" nil)
        ("normalem" "ulem" t)
        ("" "amsmath" t)
        ("" "textcomp" t)
        ("" "amssymb" t)
        ("" "capt-of" nil)
        ("" "hyperref" nil)))

The documentation for org-latex-default-packages-alist suggests that it shouldn't be changed because it's used for more than one backend. How it's used for non-LaTeX backends, I don't know. I may need to revisit this approach if I encounter problems elsewhere; so far, I haven’t.

Finally, update org-latex-classes to include the big wodge

Marginal note:Yes, I do have A4 paper. Because it’s better.

of preamble that I want:

(setq org-latex-classes
\\setmainfont{ETBembo RomanOSF}
\\setmonofont[Scale=MatchLowercase]{Operator Mono SSm}
("\\section{%s}" . "\\section*{%s}")
("\\subsection{%s}" . "\\subsection*{%s}")
("\\subsubsection{%s}" . "\\subsubsection*{%s}")
("\\paragraph{%s}" . "\\paragraph*{%s}")
("\\subparagraph{%s}" . "\\subparagraph*{%s}"))

("report" "\\documentclass[11pt]{report}"
("\\part{%s}" . "\\part*{%s}")
("\\chapter{%s}" . "\\chapter*{%s}")
("\\section{%s}" . "\\section*{%s}")
("\\subsection{%s}" . "\\subsection*{%s}")
("\\subsubsection{%s}" . "\\subsubsection*{%s}"))

("book" "\\documentclass[11pt]{book}"
("\\part{%s}" . "\\part*{%s}")
("\\chapter{%s}" . "\\chapter*{%s}")
("\\section{%s}" . "\\section*{%s}")
("\\subsection{%s}" . "\\subsection*{%s}")
("\\subsubsection{%s}" . "\\subsubsection*{%s}"))))

The [DEFAULT-PACKAGES], [PACKAGES], and [EXTRA] strings are place holders for additional customizations that can be added on a per-file basis.

I’ve chosen slightly different fonts than Peter, I increased the space above listings, and I added the parskip package because I prefer flush-left paragraphs with spaces between them.

I haven’t actually changed anything about the report or book styles yet because I’m not using them.

The resulting document looks less academic and less “computer modern”.

Custom output refined

I have no doubt that I’ll be refining things as I go, but this got me started. Thanks, Peter! And thank you, Michael for the follow-up on Twitter! (And thanks to several folks who replied to my message on the Org mailing list.)

On notes

Better meetings achieved. Maybe. Better recorded meetings, definitely.

Volume 4, Issue 1; 01 Jan 2020

Better meetings achieved. Maybe. Better recorded meetings, definitely.

I’m not a regular purveyor of New Year’s resolutions. Last year, however, I did resolve to take more notes: in particular, minutes at every meeting I attended. I am pleased to say that I was very successful. Discounting daily standups and “all hands” meetings, which I didn’t bother to try to scribe, I think I only missed a couple.

What’s more, I was more habitual about writing things down. I even made an attempt at a short “daily journal”. I should review that, but I have to finish this post fairly quickly if I want it dated 1 January!

In total, I have 219 notes for calendar year 2019 (291 if you include the daily journal ones and miscellaneous others.)

The other resolution I made was to use Org-mode to keep track of all of my “todo” items. I think I was largely successful here as well. There are 1,430 items marked “done” in my collection, 954 marked “todo”, 238 marked “canceled”, 25 marked “waiting”, 8 marked “someday” and probably a few one-offs here and there.

I should be clear, these are the “to do” things that aren’t recorded elsewhere. I’m not making an entry for every JIRA item I work on, every GitHub issue, etc. Perhaps I should, it would certainly be interesting.

What I haven’t developed yet from this exercise is a good set of habits. Ideally, as I understand this kind of thing, everything should go through that list and I should be actively curating that list. In reality, on Christmas I decided to refactor my entire Emacs init

Marginal note:It’s a literate Org file now, baby!


That wasn’t on the list. There are probably lots of occasions where I’ve decided to pursue something “off list”.

On the whole, though, I’m pleased with where I got to, and I will continue to try to improve the systems.

XML Unicode update

Some small improvements to my Emacs package for entering Unicode characters in Emacs.

Volume 3, Issue 33; 25 Nov 2019

Some small improvements to my Emacs package for entering Unicode characters in Emacs.

A few days ago, Dave asked me a question about XML and Unicode and Emacs. The precise question isn’t really relevant, it was simply the impetus to go off and look at XMLUnicode again.

Why bother, you may ask, now that Emacs has Unicode support in insert-char? It’s a fair question. Certainly, this code is a lot less significant than it once was. I still think it beats out the built in support on two grounds:

  1. If the character isn’t displayable, my code will insert a numeric character reference instead. Ok, that’s only interesting in a markup context, but that’s like 90% of my life, so it’s still interesting to me.
  2. If you’re an old XML hack and still remember that &rarr; is the ISO entity name for , you can search for that or, with a little configuration, just literally type “&” “r” “a” “r” “r” “;” and the right thing will happen.

If that seems compelling, here’s what I improved:

  1. I’ve updated the list of characters from the Unicode 3.1 list to the 12.1.0 list. I’ve also added a script so you can build the data for any version you’d like.
  2. I figured out how to test if a character can be displayed or not. That allowed me to get rid of the “list of undisplayable characters”.
  3. I tweaked the utility function xmlunicode-character-list so that you can specify a range if you want. (It still defaults to the BMP.)
  4. I separated the Helm-related functions into a separate file. They need to (require 'helm) and I didn’t want to add helm as a dependency for everyone.

Along the way, I made what felt like about a dozen false starts, errors, and broken releases. Because the smallest, simplest changes are always the hardest.

Use a function, Luke

Sometimes I’m reminded how nice Emacs lisp is, mosty because it’s lisp.

Volume 3, Issue 31; 05 Nov 2019

Sometimes I’m reminded how nice Emacs lisp is, mosty because it’s lisp.

I’ve been playing with org-brain. I wanted notes to use in the experiment, so I converted some data that I’d been storing elsewhere into Org mode. Some of those notes are about places, with geospatial coordinates like this: 37.506805,-122.24738.

I wanted to make those into OpenStreetMap links. I spent a few minutes trying to work out whether it would be easier to do this in Emacs by writing some elisp or just doing it with a script. (Well, there’s no question it’d be easier for me with a script, but I like to keep my hand in elisp programming.)

Org Mode has a handy syntax for links. This markup:

Check out [[][my homepage]].

Is rendered in my emacs buffer as:

Check out my homepage.

And a quick C-c C-l on that link will open it up in a browser.

What’s even better is that I can configure shortcuts. For example, I can configure it so that [[bug:12345]] renders as “bug:12345” and links to bug 12345 without me having to type the whole URI. Here’s a configuration that makes [[dd:docbook]] a shortcut for

("dd" . "")

The OpenStreetMap syntax I want to generate is:{LAT}&mlon={LNG}&zoom={ZOOM}

which isn’t just the coordinates stuck on the end or injected in the middle. It’s going to be necessary to parse the coordinates to generate the link.

That’s when it struck me, in lisp, code and data are delightfully intertwingly. In many cases, where you can put a string can also put a function. And this is one of those places! This configuration:

("geo" . geo-to-openstreetmap)

And this function:

(defun geo-to-openstreetmap (coords &optional zoom)
  "Parse COORDS into an OpenStreetMap link. The format of COORDS
is lat,long[/znum]. If ZOOM is specified, it overrides znum."
  (let* ((geo (split-string coords "/" t "[\s]+"))
         (latlong (split-string (car geo) ",[\s\t]*" t "[\s]+")))
    (concat ""
            (car latlong)
            (car (cdr latlong))
            (if zoom
              (if (cdr geo)
                  (car (cdr geo))

Does the trick. Now [[geo:37.506805,-122.24738][OpenStreetMap]] is a link: OpenStreetMap.

Omitting overlayed glyphs in nxml-mode

A small tweak to XML editing in Emacs.

Volume 3, Issue 8; 12 Mar 2019

A small tweak to XML editing in Emacs.

I really like nxml-mode

Marginal note:I understand that some folks still prefer psgml-mode for it’s much larger set of key bindings to manipulate elements. I have occasionally investigated porting those, but I’ve never manged to get very far. If someone with Emacs experience wants to do the XML community a solid, that’d be nice.

for editing XML (in Emacs, of course), and I have done for a while. I’m happy using RELAX NG grammars for my XML vocabularies and, for those still using DTDs, I’ll observe that it’s straightforward and mechanical to convert DTDs to RELAX NG. (As an aid to editing, I mean, even if you prefer to maintain the schemas as DTDs.)

One nice feature of nxml-mode is that it puts in overlays for numeric character references. If you stumble across a &#x263A; in a document, nxml-mode will helpfully put a “☺” after it as an overlay.

One really annoying feature of nxml-mode is that it does this for &#10;. They aren’t exactly common, but they do turn up in <xsl:text> elements with enough regularity that I find the overlayed, literal newline characters significantly distracting.

Herewith, a fix:

(defcustom nxml-char-ref-display-glyph-omit '(10)
  "A list of Unicode code points. The overlay character displayed
after numeric character references will be omitted for these values."
  :group 'nxml
  :type '(repeat integer))

(defun nxml-char-ref-display-extra (start end n)
  (when nxml-char-ref-extra-display
    (let ((name (or (get-char-code-property n 'name)
                    (get-char-code-property n 'old-name)))
          (glyph-string (and nxml-char-ref-display-glyph-flag
                             (not (member n nxml-char-ref-display-glyph-omit))
                             (char-displayable-p n)
                             (string n)))
    (when (or name glyph-string)
      (setq ov (make-overlay start end nil t))
      (overlay-put ov 'category 'nxml-char-ref)
      (when name
        (overlay-put ov 'help-echo name))
      (when glyph-string
        (overlay-put ov
                     (propertize glyph-string 'face 'nxml-glyph)))))))

I haven’t (yet) investigated getting this change pushed upstream, I’m just sharing a little itch scratching at the moment.

Reproducible research

I guess you’re getting a series of posts about Emacs, whether you want them or not. This one is about “reproducible research” or, more specifically, about making prose about code easier to write.

Volume 3, Issue 7; 12 Mar 2019 (updated 07 Jan 2023)

I guess you’re getting a series of posts about Emacs, whether you want them or not. This one is about “reproducible research” or, more specifically, about making prose about code easier to write.

Research is the process of going up alleys to see if they are blind.

Marston Bates

I’m not a researcher in any official capacity, but I am one in the casual sense that I write a lot of documents that have data embedded in them, often snippets of code.

Most recently, I was explaining how to setup authentication with externally signed certificates. There’s a lot of explanation about what has to be setup and how, interspersed regularly with code snippets that the reader is expected to run to get answers that will be used in subsequent code snippets. (And eventually, of course, to be automated so that human error is reduced, but the first step is explaining the process.)

Imagine how you might do this. Open up your editor window and next to it, open up a window to a REPL somewhere. Type in the first code snippet, try to run it until you’ve fixed all the typos, copy and paste it into your document, run it, copy and paste the results into your document. Edit the code. Repeat ad nauseum.

The techniques of reproducible research (which will look familiar to anyone familiar with Literate Programming) offer a much easier and better approach.

Babel is part of Org. It allows you to embed source code of many different kinds

Marginal note:I’ve got mine configured for more than thirty languages including Emacs Lisp, Python, Perl, LaTeX, Org, Ruby, Scala, and the shell.

directly in your document (with syntax highlighting), run it directly from your document, and automatically insert the results into your document.

I wrote ob-ml-marklogic as a plugin for Babel to support embedded MarkLogic XQuery, JavaScript, and SPARQL. Despite the fact that I implemented it a couple of years ago, it wasn’t until I turned my attention back to org-mode that I picked it up again in earnest.

The externally signed certificates example is too complicated for a weblog posting, and also, I will be quite content if I die without ever having to think about it again.

Consider instead the Fibonacci sequence. I might write a little XQuery code to compute the “nth” number like this:

declare variable $n as xs:integer external;
let $phi := (1 + math:sqrt(5)) div 2.0
  round(math:pow($phi, $n) div math:sqrt(5))

Except that’s not very interesting because in the weblog post, it’s just a code example. Here’s what it looks like in Emacs:

Org Babel Example

Everytime I hit C-c in the source code, it’s evaluated and the results are inserted into my document. If computing the 10th (yes, that HEADER line defines a variable; I can change that and hit C-c to get different results) Fibonacci number doesn’t seem like a very exciting thing to do, consider embedding the results of shell scripts, or Python, or probably your favorite language.

It’s a huge step forward in writing prose that leverages code examples, if you ask me.

Emacs for (even more of) the win

I use Emacs every day. I rarely notice it. But when I do, it usually brings me joy.

Volume 3, Issue 5; 01 Mar 2019

I use Emacs every day. I rarely notice it. But when I do, it usually brings me joy.

If you are a professional writer…Emacs outshines all other editing software in approximately the same way that the noonday sun does the stars. It is not just bigger and brighter; it simply makes everything else vanish.

Neal Stephenson

I’ve been using Emacs for well over twenty years. I use it for writing almost anything and everything (I edit Scala and Java in IntelliJ). I read my email in it. If it can be done in Emacs, that’s where I prefer to do it.

Although I’ve used Emacs for literally decades, I realized around the new year that very little about my use of Emacs had changed in the past decade or more. New editing modes had come along, of course, I’d picked up a package or two, and I did adopt Helm a few years ago, but mostly it just did all the heavy lifting that I required of it, day in and day out without complaining or getting in my way. On the one hand, that’s a testament to how good it is. On the other hand, that’s an invitation to dig in and see what I’ve missed.

At about the same time, I resolved to improve several aspects of my work life:

  • Better meeting management. I’m lead on a couple of projects at work and those projects have meetings, both regularly scheduled and ad hoc; some of them I run, some of them, I only attend.

    I realized I’d become sloppy about my participation in meetings. It’s all too easy sit in a room where there’s a meeting going on but actually read email and work on other items. (I strongly oppose the “no laptops” rule in meetings, but that’s a topic for another day.)

    There are a couple of problems with sloppy participation. First, it’s disrespectful to the person who convened the meeting and the other participants. That’s actually sufficient reason not to do it, but I think there’s another problem: it disguises the cost of meetings.

    If you’re in a meeting but also answering your email and maybe fixing a bug, then that meeting didn’t cost anything (or as much). If meetings are cheap, then there will be more of them.

    I want fewer, shorter meetings. I don’t want to disguise their cost, I want them to be perceived as damned expensive and to be avoided unless absolutely necessary.

    Sometimes, they are absolutely necessary. And I appreciate that a quick meeting can sometimes resolve an issue quickly. But if I have ten short meetings a day, let’s not pretend that I’m getting anything else productive accomplished.

    I resolved to take notes at all the meetings I attend. I’m not offering to take minutes, necessarily, but I am taking minutes of a sort. It keeps me focused on the meeting and not catching up on other things.

  • Better time management. There are lots and lots of things that I need or want to do, both professionally and personally. I’ve historically kept track off some of them in issue lists, some in saved email threads (in Emacs and Gmail, for slightly different types of reminders), in my calendar, on “todo lists” of various sorts on my phone, and on little scraps of paper. And probably other places as well.

    I resolved to keep them all in one place. Not because I think there’s one place that’s uniformly best or better, but because I hope to accomplish two things. First, by having them all in one place, I hope to be able to develop a better and more holistic view of where I’m putting my energies. Second, because I want to develop a habit

    Sidebar note:n. “A settled or regular tendency or practice, especially one that is hard to give up.”

    of recording, tracking, and preserving them.

  • Better accountability. If you work in certain science or engineering disciplines, you will have developed the habit of keeping a lab notebook. Alas, I did not. But I resolved to do so.

    I’m not interested in the legal aspects that encourage bound pages or scribing only in permanent marker. What I’m interested in is developing the habit of keeping a record. My goal is to have a place to jot down ideas and design sketches and the like. If I have sudden inspiration or if I think of an edge case that isn’t in the test suite, I want my instinct to be to write it in my journal instead of scribbling it on a scrap of paper or promising myself that I’ll remember it.

This confluence of resolutions led me quickly and more-or-less directly to Org. There is a large, active, and loyal community of Org users. I’ve played with it in the past (I even wrote about it, at least in passing, a couple of years ago) and I tinkered long enough to integrate MarkLogic into it. (Boy has that paid off in the last week or two!)

But I never used it.

I am now using it. I take minutes in it, I record all of my todo items in it, and I keep a journal in it. I’m not sure there’s much value in me attempting to wax eloquent about it or enumerate all its features, you’ll find plenty of either with a quick web search.

If you use Emacs, you should be using Org. If you don’t use Emacs, I’m confident you wouldn’t be the first person who started because of Org. It does a lot. It takes a little time to learn your way around and remember the shortcuts, but I think it’s worth it. (And if you carry an iOS device in your pocket, I recommend beorg for recording items while you’re on the go.)

Naturally, I worked out how to get XML out of it

Marginal note:“Worked out” sure is a funny way to spell “hacked together in elisp.”

. And from there, how to turn it back into the markup my weblog expects (and do so at the push of a button in Emacs, of course). So this is the first posting written in Org. It won’t be the last.

P.S. Happy birthday little weblog.

On meetings

Fewer meetings, please. But if I can’t have fewer, can I have better?

Volume 3, Issue 2; 25 Jan 2019

Fewer meetings, please. But if I can’t have fewer, can I have better?

A meeting without minutes didn’t happen.

It is a well known fact that we spend too much time in meetings. We have staff meetings, team meetings, grooming meetings, planning meetings, 1:1 meetings, review meetings, quarterly meetings, lead’s meetings, off-site meetings, standup meetings, annual meetings, and ad hoc meetings. Plus a whole lot more meetings. My favorite is the “pre-meeting planning meeting.”

I don’t know what to do about that. But I have a plan, of sorts.

One of my goals for 2019 is to develop new, productive habits. Yes, I recognize that the road to hell is paved with plans like these and I’ve probably had these plans before. But this year, I’m taking some concrete steps. They invove Emacs, of course.

The first is time management. This isn’t strictly about meetings, but it’s relevant, I promise. I have resolved that all “todo” items will go in exactly one place: Org-mode. No more scraps of paper, reminders in email, notes on my phone, weird calendar appointments, etc. I’m a little concerned that this will throw in to sharp relief just how many too many things I believe I can do. But that’s probably something I should own up to and deal with.

That puts me in daily (hourly, most days) contact with Org-mode. Which is relevant, I promise (again).

I’m not a huge fan of the “closed laptop” meeting rule. I understand what motivates it, it’s all too easy become distracted and “zone out” of a meeting. You lose focus, glance at a web page, check your email, and before long, you’re completely ignoring the meeting. But here’s the thing: if you can ignore the meeting, why are you there?

The problem with the closed laptop rule for your average technical meeting is that participants need access to bug reports, technical specs, email threads, calendars, etc.

My resolution with respect to meetings is this: I will take notes. By which I really mean, take minutes. With a few exceptions (daily standups, for example), if I’m at a meeting, I’m going to take minutes.

If you’re taking minutes, you have to pay attention. You have to stay engaged in the meeting. That’s respectful to your colleagues. It also means that nothing else is getting done. Meetings suck productivity out of a day; we shouldn’t be concealing that fact. No more fixing bugs during meetings. No more support case emails. No more backlog triage.

And, to make good on my aforementioned promise, taking minutes involves Org mode too. Because that’s where I’m going to take them. I have used lots of different techniques over the years (IRC (I miss Zakim), plain text files, markdown files, Org mode, XML files, and HTML files, at least). On balance, Org mode offers a great deal of power for a minimum of fuss.

And it’s Emacs-native. After a couple of weeks, I realized that I was doing a fair bit of repetative, manual labor: calculating next meeting dates, copying action items from previous minutes, etc. (I’m also using Org mode to maintain the agendas for meetings that I run.) So I wrote

Marginal note:If you’re not using a programmable editor, may I encourage you to make a goal of your own for 2019?

a hundred or so lines of elisp to automate those parts. Man, that’s sweet.

Here, for the curious, is my template. As I look at it “out of context,” I realized it’s tailored a bit to the meetings that I run, where I set both the agenda and take the minutes. If I’m taking notes for someone else’s meeting, some of this just gets deleted.

The template doesn’t actually include things like the title and the meeting dates; that’s generated by (or prompted for by) the elisp code. I’ve filled in some sample data in the template below to make the example more comprehensible. (It’s also worth noting that I delete some of the metadata if I email the minutes to anyone; tags, startup options, meeting numbers, etc. are just for me.)

#+TITLE: Meeting Title, 24 January 2019
#+DATE: 2019-01-24
#+AUTHOR: Norman Walsh
#+FILETAGS: :Tags:For:This:Meeting:
#+STARTUP: showeverything
#+MEETING: 004

* Minutes

** Administrivia

   + Roll call:
     + Present:
   + Scribe: Norm
   + Accept the agenda?
     + Accepted
   + Accept the minutes of the previous meeting: 17 January 2019?
     + Accepted
   + Next meeting: 31 January 2019
     + Regrets:

** Technical agenda
   + Review of open action items
   + Topic 1
   + Topic 2
   + Topic 3
   + Any other business

** Review of open action items

   + ACTION 003-01: Norm to do something.
     + Completed, 22 January 2019
   + ACTION 002-04: Someone Else to do some other thing

** Summary of new action items

** Topic 1

** Topic 2

** Topic 3

** Any other business

** Adjourned

Actions taken during the meeting occur under the various topics, but I copy them to the “summary of new action items” section and make sure they have IDs after the meeting.

As far as running meetings go, I say you can’t be too ruthless. Send an agenda beforehand. Start them on time. Stick to the agenda. If you can finish an hour-long meeting in 25 minutes, you win 35 minutes of your life back. And remember, if you don’t publish minutes, your meeting didn’t happen.

(A shout-out to the chairs and scribes of all those meetings I attended at the W3C. Most of what I learned about running an efficient meeting, I learned from you. Some of you anyway. 😁)