Showing 212 to 215

Bayesian structural time series

Bayesian methods

The Bayesian structural time series (BSTS) model is a statistical technique used for feature selection, time series forecasting, nowcasting, inferring causal impact and other applications. The model is designed to work with time series data.

Difference-in-differences models[1] and interrupted time series designs[2] are alternatives to this approach.

The model consists of three main components:

  1. Kalman filter. The technique for time series decomposition. In this step, a researcher can add different state variables: trend, seasonality, regression, and others.
  2. Spike-and-slab method. In this step, the most important regression predictors are selected.
  3. Bayesian model averaging (ensemble learning). Combining the results and prediction calculation.
Created (4 years ago)

Sensitivity analysis

Bayesian methods

I've followed this approach in much of my own applied work, using noninformative priors and carefully avoiding the use of prior information in the final stages a statistical analysis. But that can't always be the right choice. Sometimes (as in the sex ratio example above), the data are just too weak—and a classical textbook data analysis can be misleading. Imagine a Venn diagram, where one circle is "Topics that are so controversial that we want to avoid using prior information in the statistical analysis" and the other circle is "Problems where the data are weak compared to prior information." If you're in the intersection of these circles, you have to make some tough choices! More generally, there is a Bayesian solution to the problem of sensitivity to prior assumptions. That solution is sensitivity analysis: perform several analyses using different reasonable priors. Make more explicit the mapping from prior and data to conclusions. Be open about sensitivity, don't try to sweep the problem under the rug, etc etc. And, if you're going that route, I'd also like to see some analysis of sensitivity to assumptions that are not conventionally classified as "prior." You know, those assumptions that get thrown in because they're what everybody does. For example, Cox regression is great, but additivity is a prior assumption too! (One might argue that assumptions such as additivity, logistic links, etc., are exempt from Fisher's strictures by virtue of being default assumptions rather than being based on prior information—but I certainly don't think Mayo would take that position, given her strong feelings on Bayesian default priors.)

–Gelman

What links here

Created (4 years ago)

Empirical Bayes

Bayesian methods

Also known as maximum marginal likelihood

Empirical Bayes methods are a way of setting priors by looking at the data.

Empirical Bayes may be viewed as an approximation to a fully Bayesian treatment of a hierarchical model wherein the parameters at the highest level of the hierarchy are set to their most likely values, instead of being integrated out.

Created (4 years ago)

Eshell

C-c RET🔗

This takes some getting used to. Suppose you type this: 

mkdir /home/user/some-very-long-name

press C-c RET, then what you have on the prompt is this (I put in a vertical bar to represent the point/cursor): 

mkdir /home/user/some-very-long-name ⋮ mkdir /home/user/some-very-long-name

so you can delete the second mkdir and type && cd or something.

Once you're used to this sort of feature, you'll always be looking for equivalents in other shells or feel handicapped. You could write your own Rosetta stone: a cheatsheet for equivalents in fish, oh-my-zsh, oh-my-bash, or whatever other shell you switch to in the future. All skills you learn in eshell are transferable.

The important thing is that you explore the most powerful features of whichever shell you're in so you know about them and know they're possible. That's how you grow your skill, and become able to exploit the potential of exotic shells like a riced-out Xonsh or Ambrevar's SLY-on-steroids. It builds up.

My best Eshell hotkeys🔗

  • C-c M-o – eshell-mark-output
  • C-c C-p – jump to previous prompt
  • C-M-l – eshell-show-output, almost the same, but jump to top of output
  • C-c RET – more on this later
  • C-c C-u – like C-u on a terminal obeying Bash hotkeys
  • C-c M-d – eshell-toggle-direct-send – lets you say y to y/n prompts without pressing enter, I think

Some of my really nice custom bindings:

  • C-M-j – my-insert-other-buffer-file-name
  • C-c C-l – my-counsel-eshell-history
  • <escape> – crux-switch-to-previous-buffer

The Eshell language🔗

You can type elisp expressions without outer parens, like 

λ kbd "C-x f"

or 

λ define-key

Try it! it returns 

usage: define-key: (KEYMAP KEY DEF)

It's a bit like an alien fish. Oddly similar because in both shells, parens delineate nested function calls. The equivalent in Bash would be using grave accents, e.g. echo `concalc 1+2`. Try 

λ echo (+ 1 2)
λ define-key $global-map (kbd "C-x t") #'crux-transpose-windows
λ list-timers

So cool! You really should try. If you don't have an Emacs open, open it and type M-x eshell. Type these commands out by hand.

More things to try: 

λ concat a b
λ setq a 2
λ + a 3

Hear this:

Unlike regular system shells, Eshell never invokes kernel functions directly, such as ‘exec(3)’. Instead, it uses the Lisp functions available in the Emacs Lisp library. It does this by transforming the input line into a callable Lisp form.

[2024-04-08 Mon] That it just does lisp becomes obvious if you type a command that doesn't exist. Compare it in eshell versus shell.

To see the Lisp form that will be invoked, type: 

λ eshell-parse-command "echo hello"
λ eshell-parse-command "echo hello; echo multiple words; setq a 2 b 3"

Also 

λ which ls

One use I've for Eshell that's not possible in any other shell: after editing ~/.profile or ~/.xsessionrc or similar files, I can make it apply to the running emacs environment with simply 

λ . ~/.profile

This is not possible in any other shell because they're external processes, whereas Eshell's "process" is the Emacs process.

May not seem important to source ~/.profile, but I feel there's something about the fact you can that proves there's no other sane shell choice. Emacs is supposed to be your HCI (human-computer interface). Why create roadblocks?

By the way, so you know, . .profile is is not the same as source .profile (in plain Bash, it is the same).

If you're looking for how to do something, try the info manual at C-h i d m Eshell <RET>. They tell you how to do for loops and whatever else you need.

Eshell with timestamped prompts, automatically timing commands, and backreferences!🔗

(defun my-eshell-timestamp-update ()
    "When added to `eshell-pre-command-hook', the first string --:-- in the
  prompt becomes a timestamp like 13:59 after you run a command."
    (save-excursion
      (forward-line -1)
      (when-let* ((unfilled-timestamp "--:--")
                  (end (search-forward unfilled-timestamp nil t))
                  (beg (- end (length unfilled-timestamp)))
                  (inhibit-read-only t))
        (delete-region beg end)
        (insert (format-time-string "%H:%M"))
        (add-text-properties beg (point) '(font-lock-face eshell-prompt)))))

(defvar my-eshell-last-cmd-start nil)

(after! eshell
  ;; Always time commands that take longer than 1 second.
  (add-hook 'eshell-pre-command-hook
            (defun my-eshell-time-cmd-1 ()
              (require 'ts)
              (setq my-eshell-last-cmd-start (ts-now))))
  (add-hook 'eshell-post-command-hook
            (defun my-eshell-time-cmd-2 ()
              (when my-eshell-last-cmd-start
                (let ((n (ts-diff (ts-now) my-eshell-last-cmd-start)))
                  ;; Workaround for bug(?) that makes post-command-hook run
                  ;; after noops but pre-command hook doesn't run before. By
                  ;; nulling this we'll know next time if a real command did
                  ;; run, since it won't be nil anymore.
                  (setq my-eshell-last-cmd-start nil)
                  (when (> n 1)
                    (let ((inhibit-read-only t)
                          (beg (line-beginning-position)))
                      ;; TODO: position point correctly regardless
                      ;; of other post-command-hooks that may have run
                      (goto-char beg)
                      (insert
                       "Command finished, "
                       (if (> n 100)
                           (number-to-string (round n))
                         (let ((s (number-to-string n)))
                           (string-match (rx (* digit) "." digit) s)
                           (match-string 0 s)))
                       " s elapsed.\n")
                      (add-text-properties beg (point) '(font-lock-face eshell-prompt))
                      (goto-char (line-end-position))))))))

  ;; Custom prompt
  (add-hook 'eshell-pre-command-hook #'my-eshell-timestamp-update)
  (setq eshell-prompt-regexp
        (rx (or (seq bol "Command finished " (* nonl) eol
                     (* nonl) "] λ ")
                (seq bol (* nonl) "] λ ")))
        eshell-prompt-function (lambda () (concat "[--:--] λ ")))

  ;; Backrefs!
  (defvar my-eshell-backref-counter 0)
  (make-variable-buffer-local 'my-eshell-backref-counter)
  (add-hook 'eshell-post-command-hook
            ;; NOTE: bizarrely, variables made this way won't show up in
            ;; describe-variable, but echo $vX still works.
            (defun my-eshell-save-output-into-backref ()
              (let ((output (buffer-substring (eshell-beginning-of-output)
                                              (eshell-end-of-output))))
                (set (make-local-variable
                      (intern (format "v%d" (cl-incf my-eshell-backref-counter))))
                     output))
              (eshell-previous-prompt 1)
              (goto-char (line-beginning-position))
              (let ((beg (point))
                    (inhibit-read-only t))
                (insert "v" (number-to-string my-eshell-backref-counter) " ")
                (add-text-properties beg (point) '(font-lock-face eshell-prompt)))
              (eshell-next-prompt 1)))

What links here

Created (4 years ago)
Showing 212 to 215