2022-May-25
00:48
Your deferred or async I/O chains may not need librariesđ
UPDATE: I have written the package github.com/meedstrom/asyncloop based on this post.
Deferred, aio, async/await: I struggled to learn their concepts for my use case. Why did I need to understand so much? Turns out they were just overengineered for my specific use case, and I suspect it's a common use case.
Suppose you just want to slice up a compute-heavy chunks of Elisp into imperceptible pieces or "stages", to keep Emacs feeling snappy:
(defun stage-1 () ...) (defun stage-2 () ...) (defun stage-3 () ..)
How to run these in sequence, but have Emacs listen to user input events in between? Here's one way that embarrassingly took me four years to come up with:
(defvar my-pipeline '(stage-1 stage-2 stage-3 ...)) (defun stage-1 () ... (run-with-idle-timer .3 nil (pop my-pipeline))) (defun stage-2 () ... (run-with-idle-timer .3 nil (pop my-pipeline))) (defun stage-3 () ... (run-with-idle-timer .3 nil (pop my-pipeline)))
You could even use anonymous lambdas, if you like:
(defvar my-pipeline) (defvar my-pipeline-master-copy #'((lambda () ... (run-with-idle-timer .3 nil (pop my-pipeline))) (lambda () ... (run-with-idle-timer .3 nil (pop my-pipeline))) (lambda () ... (run-with-idle-timer .3 nil (pop my-pipeline))))) ;; Get the ball rolling. (funcall (pop (setq my-pipeline my-pipeline-master-copy)))
Looping through data
What about an alternative to deferred:loop
, to work through a list of data piecemeal? Same basic idea:
(setq some-list '("bob@foo.com" "bill@foo.com" "dana@foo.com" "jane@foo.com" ...)) (defun work-on-that-list (that-list) (let ((addr (pop that-list))) (send-spam-mail addr)) (when that-list (run-with-idle-timer .3 nil #'work-on-that-list that-list))) ;; Get the ball rolling. (work-on-that-list some-list)
Aside
Aside: What about a "try-catch-retry" pattern, to keep running in the face of random keyboard-quit events (C-g
) from the user?
First, I regret that I cannot upscale the font with which to write this:
NO.
Keyboard-quit events must successfully stop things. None of us knows better than the end user. Consider writing your code such that it's fine for the pipeline to be interrupted and not restarted until the next time it would've been naturally invoked.
But if you must, here's a second repeating timer (and here the package named-timer
becomes really nice for bug prevention). Note that we also redefine the function from before to demonstrate that any time you signal an error you may as well cancel the extra timer while you're at it (you don't want a new error every 10 seconds!).
;; (This 50-line library prevents dozens of bugs and oughta ;; enter emacs core.) (require 'named-timer) (named-timer-run 'my-chain-restart 0 10 (lambda () (if some-list (work-on-that-list some-list) (named-timer-cancel 'my-chain-restart)))) (defun work-on-that-list (that-list) (let ((item (pop that-list))) (send-spam-mail item) (when OH-NO-SOMETHING-LOOKS-WRONG (named-timer-cancel 'my-chain-restart) (error "Error encountered working on: %s" item))) (when that-list (run-with-idle-timer .3 nil #'work-on-that-list that-list)))
With that out of the way, what about interweaving the two kinds of threads? That is, the straightforward pipeline from before, but where one of its steps is meant to run repeatedly in a loop?
(defvar my-pipeline #'(do-things work-on-that-list do-other-things)) (defvar some-list '("foo" "bar" "baz" "fnord" "gjihjgk")) (defun work-on-that-list () ;; now it must be a no-argument function (when that-list (let ((item (pop that-list))) (send-spam-mail item)) (if that-list ;; still things in the list (named-timer-run 'my-chain .3 nil #'work-on-that-list) (named-timer-run 'my-chain .3 nil (pop my-pipeline)))))
Immediate launch
Let's take a detour and look back on the first snippet in this page, to keep things simple.
What if, most of all, we want to launch a chain immediately, even though we know the user just did something (so idle time is zero)?
(Launching immediately can be useful for example when you'd prefer to compute something right as the minibuffer gains focus, a situation tight with time pressure before the user starts typing.)
Now we can't use idle timers, but we can use something even simpler: check that current-idle-time
is always higher at the end of each function than at the start.
(setq my-pipeline #'(stage-1 stage-2 ...)) (defun stage-1 () (let ((T (current-idle-time))) ... (when (time-less-p (current-idle-time) T) (setq my-pipeline nil)))) (defun stage-2 () (let ((T (current-idle-time))) ... (when (time-less-p (current-idle-time) T) (setq my-pipeline nil)))) (while my-pipeline (funcall (pop my-pipeline))
Now the pipeline will run all at once, without delay, unless the user does something, in which case it bails out to avoid blocking Emacs.
Now, what if we want to do that but gracefully fall back to idle timers, so that the pipeline will still be guaranteed to run to completion?
(defvar my-pipeline-last-idle-value) (defvar my-pipeline) (defvar my-pipeline-template #'(stage-1 stage-2 stage-3 ...)) (defun my-pipeline-chomp () (when my-pipeline (funcall (pop my-pipeline)) (if (time-less-p my-pipeline-last-idle-value (or (current-idle-time) 0)) ;; If user hasn't done anything since pipeline start, go go go. (progn (setq my-pipeline-last-idle-value (or (current-idle-time) 0)) (named-timer-run 'my-chain 0 nil #'my-pipeline-chomp)) ;; Otherwise go slow and polite. (cl-incf my-pipeline-last-idle-value .3) (named-timer-idle-run 'my-chain .3 nil #'my-pipeline-chomp)))) ;; It'll be easier to reason about if we have a separate function for ;; (re)starting. Adapting `my-pipeline-chomp' to also be able to restart the ;; pipeline would make it horribly complex. (defun my-pipeline-start () "Re-initialize and start doing whatever this pipeline is meant to." (named-timer-cancel 'my-chain) (setq my-pipeline-last-idle-value 0) (setq my-pipeline my-pipeline-template) (my-pipeline-chomp))
Finally, to include a loop in this pipeline, you know what to do: and write one of the stage-*
functions exactly like that function we named work-on-that-list
, and it Just Works.
(defvar some-list '("foo" "bar" "baz" "fnord" "gjihjgk")) (defun work-on-that-list () (when that-list (let ((item (pop that-list))) (send-spam-mail item)) (when that-list ;; still things in the list (push #'work-on-that-list my-pipeline))))
See my package github.com/meedstrom/asyncloop.