Taking ownership of org-id

#emacs

Let's say most of your Org files sit in a folder /home/kept/notes/ but some others are outside, scattered here and there, plus you'd like to try not depending on Org-roam's M-x org-roam-update-org-id-locations all the time.

The challenges with org-id:

  1. The classic way to tell it where to look for IDs is adding the directories to org-agenda-files.
    • Unfortunately with thousands of files, this slows down the agenda something extreme. Not an option.
  2. An alternative way is to populate org-id-extra-files or org-agenda-text-search-extra-files.
    • See snippet A below.
      • Unfortunately with thousands of files, this slows down M-x customize-group for either org-id or org-agenda, to a grinding halt.
  3. To sidestep the small problem with #2, you could trust in org-id to keep itself updated, because it does that every time your Emacs creates or searches for an ID. You regenerate org-id-locations once (or well, once every time you wipe .emacs.d). See snippets B or C.
  4. org-id complains about duplicate IDs because it's also looking in e.g. the versioned backups generated by Logseq
    • So, you need some sort of exclusion ruleset.
      • For an elisp-only way, see snippets A or B.
      • A natural way is to obey .ignore or .gitignore, if you already keep such files. I've found no elisp gitignore parser, but see snippet C for a way to use ripgrep's builtin parser.
    • Why org-roam didn't give you this problem? If I'm reading the source code right, it has actually been suppressing org-id errors!

Snippet A 

;; Populate `org-id-extra-files'
(dolist (file (mapcan (lambda (dir)
                        (directory-files-recursively dir "\\.org$"))
                      '(;; Example values
                        "/home/kept/notes/"
                        "/home/kept/project1/"
                        "/home/kept/project2/")))
  (or (string-search "/logseq/bak/" file)
      (string-search "/logseq/version-files/" file)
      (string-search ".sync-conflict-" file)
      (push file org-id-extra-files)))

;; Then either run M-x org-id-update-id-locations or restart Emacs.

Snippet B 

;; Populate org-id without setting `org-id-extra-files'.  Only do it if
;; `org-id-locations' is gone.
(when (or (and (not (file-exists-p org-id-locations-file))
               (null org-id-locations))
          (if (null org-id-locations)
              (org-id-locations-load)
            (if (listp org-id-locations)
                (null org-id-locations)
              (hash-table-empty-p org-id-locations))))
  (org-id-update-id-locations
   (seq-remove (lambda (file)
                 (or (string-search "/logseq/bak/" file)
                     (string-search "/logseq/version-files/" file)
                     (string-search ".sync-conflict-" file)))
               (mapcan (lambda (dir)
                         (directory-files-recursively dir "\\.org$"))
                       '(;; Example values
                         "/home/kept/notes/"
                         "/home/kept/project1/"
                         "/home/kept/project2/"))))
  (org-id-locations-save))

Snippet C 

;; Populate org-id without setting `org-id-extra-files'. Only do it if
;; `org-id-locations' is gone.
(when (or (and (not (file-exists-p org-id-locations-file))
               (null org-id-locations))
          (if (null org-id-locations)
                     (org-id-locations-load)
                   (if (listp org-id-locations)
                       (null org-id-locations)
                     (hash-table-empty-p org-id-locations))))
  (dolist (default-directory '(;; Example values
                               "/home/kept/notes/"
                               "/home/kept/project1/"
                               "/home/kept/project2/"))
    ;; Borrow ripgrep's ability to obey .ignore/.gitignore
    (org-id-update-id-locations
     (split-string (shell-command-to-string "rg -ilt org :ID:") "\n" t))
    (org-id-locations-save)))

Bonus snippet: full reset 

;; FOR TESTING: wipe all records
;; You ONLY need to wipe if it won't shut up about duplicates!
(progn
  (delete-file org-id-locations-file)
  (setq org-id-locations nil)
  (setq org-id--locations-checksum nil)
  (setq org-agenda-text-search-extra-files nil)
  (setq org-id-files nil)
  (setq org-id-extra-files nil))

Sudden amnesia

Every time your Emacs quits unexpectedly, it can forget many ID locations! To ensure it remembers, either use a hook like 

(add-hook 'after-save-hook
 (defun my-save-id-soon ()
   (run-with-idle-timer 10 nil #'org-id-locations-save)))

or enable persist-state and add the hook 

(add-hook 'persist-state-saving-functions
          (lambda ()
            (when (fboundp 'org-id-locations-save)
              (org-id-locations-save))))

Final fix

All these issues go away now that I let org-node populate ID locations for me, via its user option org-node-extra-id-dirs.

Created (5 months ago)