Statistical Graphics and<br /> Data Visualization

Our “friends” at Statgraphics™ (Statgraphics® is a registered trademark of StatPoint, Inc) sent me unfriendly letters and threaten me “appropriate legal actions” in case I use this URL any longer, and force me to move further posts to a new site.

New posts will be at

http://statisticalgraphics.blog.com

I will give further details on their strange behavior later …
(Some software companies seem to invest more money into lawyers than into good software developers)

… is the name for subgraphs of a large network (e.g. telephone calls) which have certain target properties.

Chris Volinsky has a very nice page, which allows to look for subgraphs that connect authors by papers in computer science journals (based on DBLP) or actors connected by movies (based on IMDB).

Here is the proximity graph that connects me (not being a computer scientist) with Donald E. Knuth:

Visit Chris’ page here.

It’s been too long since the last posting (on barcharts) in the teaching corner. This one will be on histograms.

Histograms are often mistaken with barcharts. The fundamental distinction between the two is

• Barcharts show counts (or weights) for the discrete axis of a categorical variable
• Histograms show an approximation of the density function (if scaled accordingly) of a countinuous variable.

As a consequence, the only thing that can be quantified in a barchart is the bar height (better length, which makes it independent from their orientation). On the other hand, in a histogram, the area of the boxes is proportional to the density approximation. If all bars have the same width in a barcharts, or gaps are drawn in a histogram (which is complete nonsense), the two plots can get mixed up.

(% votes for Kerry in the 2004 presidential election)

Much has been written about optimal bin width for histograms - almost nothing about the choice of the anchor point. Changing the latter often changes the shape of the histogram more dramatic than choosing between 8, 9 or 10 bins.
Setting the anchor point from 0 to -2.4 yields:

(changing the anchor point to -2.4)

In most applications, there are sensible breaks which can be chosen. Since we look at 3,111 voting districts, we can use far more bins and start at 0 with bin width of 5.

(using meaningful parameters (0, 5) - density estimate added)

If we link a second attribute into the histogram, the whole thing gets more exiting!

(all districts where more than 15% have a college degree selected)

We don’t really can see what is going on here (although we might guess, that there is a slightly higher proportion of highlighting towards the higher votes for Kerry).

When we use the same normalization trick as for Spineplots (see previous post), we get the clearer picture of the Spinogram:

(highlighting in a Spinogram)

Well, that’s what we would have expected, probably except for the increase at the left end of the scale.

The problem with the histograms used so far is that we looked at voting districts, and not at voters! This will distort the impression if the districts are not of equal size. Weighting above histogram will move the mode further to the right.

(the weighted histogram represents voters not districts)

Finally we get the weighted spinogram, which probably supports more the hypothesis … of the selected group.

(the weighted spinogram)

(Sorry for the lengthy post … but concepts get a bit more complex)

Antony pointed me to this nice example found on BBC News.

So what is the message here? “Chinese and other foreigners (not being British nationals) more and more fill Brirish jails …”

Well, as we look at the percentage changes, we do not have any clue about the underlying group sizes. As whites are by far the larger group in this example, the absolute increase for whites is certainly much bigger. Any better display at hand?

So called “Skyline Plots” - as implemented in RENOIR - take the absolute size of groups into account by adjusting the bin width, such that the plot covers both aspects: absolute and relative change.

(This is certainly a different example as the one from BBC News, but without the absolute figures it is impossible to recreate the skyline plot for the prison example.)

Looking at the colors, we find the odd choice of coding an increase of prisoners in green and the decrease in red. (Does not make much sense, unless the graph comes from the company which runs the prinson …)

That’s it for this year. Au revoir 2007!.

Stage Results	cumulative Time	Ranks
(click on the images to enlarge)

Prolog: Did anyone have Thor HUSHOVD on his list?
Stage 01: Except for 5, all arrived in the peleton
Stage 02: MC EWEN already on 3.
Stage 03: First drop outs
Stage 04: BOONEN still keeps the yellow jersey
Stage 05: Only O’GRADY can’t keep the pace of the top 10 from the Prolog
Stage 06: First 37 still in a window of one minute
Stage 07: This was the day of team T-Mobile!
Stage 08: No stage to remember …
Stage 09: Still waiting for the mountains, so we look at Sebastian JOLY …
Stage 10: MERCADO and DESSEL out of the blue?
Stage 11: LANDIS now in yellow
Stage 12: POPOVYCH’s day, now on 10. 5 withdrawls after the mountains.
Stage 13: PEREIRO SIO’s second place awards him the yellow jersey.
Stage 14: No changes within the top 17.
Stage 15: LANDIS back in yellow; two more mountain stages; down to 152
Stage 16: LANDIS passes yellow to PEREIRO, KLÖDEN the real winner in the end?
Stage 17: LANDIS back after great ride; top 3 within 30”
Stage 18: No changes, we are all waiting for the show down tomorrow
Stage 19: Everything as expected, LANDIS too strong for PEREIRO
Stage 20: Profile of a winner …

To play with the data yourself, get the Mondrian software and the dataset. Thanks goes to Sergej Potapov, who wrote the script to manage the data!

Here is an example of the so called “Sectioned Density Plot”, which was recently published in “The American Statistician” (Vol 60, No. 2, 167-174).

Using a simple histogram, maybe with an added density estimator, and/or a simple standard boxplot for group comparison does the job here. No need to “invent” a new plot, which introduces more problems as it solves any.
Actually this plot makes a good case against the use of graphics …

Haven’t we been preaching against 3-d barcharts and the like for a long time? Here is what we didn’t think of in our wildest dreams: an animated 3-d barchart!


As you might guess, this is the usage statistics from this blog over the last week. It is not hard to draw “The Good” (far less sensational)

(This statistics also tells me why my quota is used up, so I moved most of the recent stuff to another server!)

Well, the Tour de France 2006 is only a few weeks ahead, probably still time to give an update on the 2005 data.
We now have the data recompiled with drop outs (thanks to Sergej).

Interesting to see that almost half of the cyclists categorized as “sprinter” don’t make it to the Champs-Elysees.

Looking at the ranks is quite funny as well. Here’s what happens when you start as first and last …
(Certainly, David Zabriskie would have probably looked better when not having the crash in the team time trial)

For those of you who do not stop by at JunkCharts regularly, here is what John S. found:

Probably one of the best example of “featurism”; computer scientists set free with no idea of application …

We got this from Friedrich who is just finishing a sabbatical in Paris:

Michael came up with the idea of a challenge. Here’s what he wrote:

Here’s a graph you may find interesting, published yesterday in Le Monde.
The manifest goal is to compare estimates of les manifestants at les manifestations according to the unions vs. the police throughout France. An admirable goal for a newspaper, and done with some sensitivity to graphic principles, in a style reminiscent of les Albums: color-coding key (but no scale), time-series sub-graph, numbers included for those who want/need them.

But what do we see, and is there anything we could see better by some other display or encoding of these numbers? Obviously, either the police in Paris can’t count or deliberately misrepresent things, or the unions there have enormously dilated pupils. Is the over-count ratio greater in Paris than in Marseilles? Were the police in Mont de Marsan all en cong’e? Is Dijon the place with the closest correspondence between the estimates? Presumably, they used count ~ area (or sqrt radius); should they have used a different power for better perceptual comparison? What is missing from this display? e.g, how did the turnout in Toulouse compare to that in Nante or Rennes, taking population into account?
best,
-Michael

You can download the data here. Any comments are welcome and of course, any attempt of a better visualization will be posted here!

Have fun!

---

Here are two posts:

Michael sent this graphics:

He wrote:
“Here is a simple log-log scatterplot that answers some of the questions posed originally. On a log scale, the relation is fairly linear, with a slope of 1.19, indicating that union estimates are on average about 154% of those by the police. I’ve labelled only those cities that have a log-residual greater than 0.15, and scaled the axes so that the line of equality is approx. 45 deg.”

Romain’s (updated) suggestion is:

(See also his comment!)