Monday, September 1, 2008

Back-of-the-Envelope Statistics

I recently gave a half-hour talk on statistics for non-experts, which sounds like a half-hour visit to hell for everybody concerned, but it was actually a lot of fun. The audience was sharp and engaged, and I think everybody got something out of the presentation.

In the talk, I was trying to achieve three things. The first goal was to give my audience a gut-level feel for random fluctuations and signal-to-noise ratios. I wanted to convey the same kind of rough, theorem-free intuition that physicists commonly use when approaching noisy data. The second and most important goal was to help my audience differentiate clearly between the "statistical significance" of a measured effect vs. the size of that effect. This is a distinction that the press almost never makes when reporting about scientific research. For that matter, even scholarly authors tend to equivocate on the different senses of the English word "significant" when it helps them further their agenda. Finally, I wanted to show people a toy model of a multiple regression, so that they could at least know what the experts are talking about when they present model coefficients or discuss "the problem of missing variables" in correlational studies.

In case anybody's interested, the Powerpoint document is here.

N.B., the presentation includes a map from American Scientist with no written explanation alongside it; that's because I spoke about the map off-the-cuff. The map is from an article by Howard Wainer that appears here; I also discuss Wainer's map in my pamphlet "Randomness and Reason", available here.


Danimal's Game

Just in case statistics isn't everybody's idea of fun, I thought I'd also take a moment here to develop a comment that Danimal made on an earlier post. Danimal suggested an interesting variation of the standard change-a-word game:
Find a route from one four-letter word to another, but making changes only between neighboring letters in the alphabet, i.e. "b" can only become "a" or "c".

For example, we are allowed to change BELL to CELL, because B and C are neighboring letters in the alphabet. But we can't change CELL to TELL, because C and T are not neighboring letters.

As Danimal pointed out, the new constraint makes it difficult to design instances of this game by hand. A well-designed game, by the way, should satisfy the demand that the starting word and the ending word share no letters in the same position. The BELL example now shows the major difficulty: how are we ever going to get away from that E? Vowels aren't adjacent to one another!

This suggests the strategy of "pivoting" on a double-vowel combination, as in the following three-letter example:

SOY [note, we now have two vowels to work with]
SPY [thus we can pivot from the O to the second vowel]

Admittedly SOX is not a very appealing word, but this was the best I could do by hand.

Using the computer, I did manage to find a (single) viable four-letter instance of Danimal's game:


(A hint is here and the solution is here.)

Graphically speaking

A fair move in Danimal's game is a fair move in the standard game, but the converse is not true. This means that Danimal's rule deletes edges from the standard connectivity diagram (over 90% of them). In other words, the adjacent-letter constraint shatters the standard game's supercontinent of words into hundreds of tiny atolls. The two largest atolls have twelve words each, as shown in the graphs below (spoiler alert!):

The movie below gives a fast review of all of the sizable atolls in Danimal's game. (A clearer version of the movie is here.)


People were kind enough not to send me any congratulatory emails last June 7th, which was roughly when I passed the halfway point of my table of American male life expectancy (see my earlier post on "The Prime of Life"). Today I'm 39 and officially celebrating the beginning of Life, Part II. Now: will it be the motorcycle or the convertible?


danimal said...

Hey Jason, I'm flattered that you took the time to look into the modified word-connection game that you generously christened as "Danimal's game".

Looking at your results and the animation, the "shattering of the supercontinent" of connected words into "isolated atolls" seems to be more complete than I'd imagined!

I'm playing around now with a modification where a second type of step between words would be allowed, as a means to hop from atoll to atoll, and forage one's way to a connection between two words.

The one I'm looking at right now is anagramming, so that in addition to the single-letter moves, you're allowed to rearrange the letters of a 4-letter word to form another valid word.

I'm in the midst of trying to get from the word "BAKE" to the word "DIME" using those two allowed steps, but without success so far...

I'll have to play with it more to see if it turns out to be a challenging game with enough possible solutions. I'm having a good time trying this first puzzle, though!

JasonZimba said...

Awesome! So it's like "island-hopping" in a way. Let us know if you come up with any fun instances of the game.