#statistics, Identifying power law data

Under at least one definition, a fat-tail distribution is one whose tail is fatter than that of the exponential distribution.

Examples

• Cauchy
• Pareto
• Zipf
• Weibull with low k parameter

All exist in Stan.

Pareto and Zipf are both simple power laws with a negative exponent, scaled so that their cumulative distributions equal 1. The difference is that Zipf is discrete.

The "80-20 law", according to which 20% of all people receive 80% of all income, and 20% of the most affluent 20% receive 80% of that 80%, and so on, holds precisely when the Pareto index is α = log₄(5) = log(5)/log(4), approximately 1.161.

80-20 also implies 64/4 and approx. 50/1 (51.2/0.8)

Cauchy has no well-defined mean.

#emacs

The command find-dired lets you generate a dired buffer out of Find results.

wdired lets you edit file names and attributes with normal buffer editing commands. You know how C-x C-q is the standard "toggle read-only" binding? Well, C-x C-q toggles wdired.

What's often not mentioned about WDired is:

• As well as renaming a file you can move it, e.g. Change the name to ../../name and it moves up two parent folders as well as changing name.
• if the file is a symlink you can just change its target too
• file attributes, sure just change them in WDired too. (Needs wdired-allow-to-change-permissions set to t)
• change the name to blank, the file will be deleted

You can use any character to mark files and then perform an operation on a specific type of mark. That's why deleting uses its own mark character. So go through lots of files marking some a and others b and yet others c and then move them all to different places.

In fact any shell command can be run on selected files using the ! command.

For subdirs, i calls dired-maybe-insert-subdir, and an alternative is dired-subtree from Fuco1's dired-hacks. But I suspect it's better to adopt a buffer-proliferate approach than to try to keep everything in one Dired buffer. One use i can see is mass renaming recursively with wdired, where you first have to insert all the subdirectories. To do that, C-u C-8 C-x C-j (any numeric arg) and add the -R switch.

Marking files: you can mark a region and then press m to mark all files within.

The m key is really a shorthand for * m, but there are other marking commands like as follows. Note that these can be a transient map / hydra.

• * m mark this file
• * * mark all executables
• * @ mark all symlinks
• * s mark everything
• * ! (shorthand: U) unmark all files
• * C-n (or M-}) move to next marked file
• * t toggle marks (colloquially, "invert selection")
• * % REGEXP mark files whose names match REGEXP
  • % m REGEXP (synonym to above)
  • % g REGEXP mark files whose contents contain a match for REGEXP

• Perceptually uniform colormap
• Linear city
• House building
• How does thermal mass make a house passive?

#statistics

Identifying power law distributions: MLE/WLS "Listen to your body"

Identifying power law tails: Hill estimator (Extreme value theory) "Let the tail do the talking"

We have been taught that the Gaussian distribution is "normal" because of the Central Limit Theorem and the Law of Large Numbers. But the CLT we're taught in school is incomplete.