We all make some wrong and costly assumptions. I wrote a blog on data visualization in Portuguese for about a year and then I had to give up, because no one seemed to care. I’m selling a tutorial on how to create Excel dashboards that I am proud of, but I should have started with a simpler version that delivers similar results (I’m working on that, by the way…).

Many of these assumptions are powered by what Chip and Dan Heath in Made to Stick call the “curse of knowledge” (”the better we get at generating great ideas—new insights and novel solutions—in our field of expertise, the more unnatural it becomes for us to communicate those ideas clearly”). Our wishful thinking makes us to believe that the knowledge gap is narrower than it really is, and some basic notions that we take for granted are not.

I’d love to write a blog on data visualization using higher-end tools like Tableau or Spotfire, but you can’t tell people “ditch Excel, use these great tools instead”. They have their (growing) market, but an overwhelming  proportion of business charts are made in Excel because that’s the only tool people have access to. Excel is good enough to teach sound visualization principles, so visualization experts should start by saying “you can do it in Excel; here is how”. At some point the newly acquired knowledge would make people move up, if needed. In information visualization, we have the (graphic literacy-wise) rich and the poor. Now we need a solid middle class. Accessible learning tools is one of the answers.

(This is what I am trying to do with pie charts: instead of banning them, I’m trying to show how to create acceptable pie charts. At some point people will realize that they will need something better. I may be wrong, but the other options don’t seem to be working, either.)

If we fail to communicate this simple message (”you can do it in Excel; here is how”) do you know what we’ll get? A new Dundas/Crystal Xcelsius user.

Post from: Jorge Camoes' Charts

Wrong. It starts by asking yourself if you really need one. Perhaps a statistical measure of some sort is good enough, perhaps you should use a table. If your job is to find patterns in a data set and build shared knowledge about it, what really matters is how efficiently the message is sent, and how efficiently it is received by the audience (two different things).

Misconception #2: You Should Master the (Technological) Tools of the Trade

No, you don’t. Just because you know how to create a chart in Excel it doesn’t mean that you know how to create a chart. If you use Microsoft Excel as your charting software then yes, you should learn more Excel (to spend more time with the kids). But you must go beyond technology, or else you end up creating some very stupid charts. Please note that a vast majority of Excel training courses will not teach you what it should (best practices). It will only tell you how to make “cool” graphs, like a 3D exploded pie chart.

Misconception #3: Defaults are good enough

They aren’t. Each chart must be tailored to the specific data set, audience and message. For instance, try to create a graph that clearly displays a large number of series and you’ll fail if you use the defaults (but can do it with clever color coding). And if you use recognizable defaults, like the Excel 2003 charts, you’ll look very, very, lazy (at best).

Misconception #4: Vendors obviously implement the very best templates

(I’ve heard this one recently, and I found it so incredibly naive that I had to write about it.) They don’t. About 90% of the Excel 2003 chart gallery is junk, and you must heavily reformat the remaining 10% to get something useful. Select other tools, like Crystal Xcelsius and the scenario is even worse. And I am unable to create in Cognos something that remotely resembles a chart (people tell me that version 8.4 is a little better).

Misconception #5: Better charts are just “prettier” charts

I hear this all the time. A good chart may look “prettier”, but that’s just an unintended consequence of a design that communicates better. In information visualization, prettiness must be a by-product of function. The very concept of a “better communicator” is sometimes difficult to comprehend, and trying to explain it is a waste of time, because people need to see it in action. You must take the user by the hand and guide him/her. You must force comparisons: “what can you learn about x using this chart?” and “what can you learn about x using that chart?” “how long did it take you to learn x using this and using that?”.

Misconception #6: It’s All About the Wow Factor

It is not. Many marketers and graphic designers fail to understand this. Marketers are hopeless in their relentless search for the wow factor and the eye-catching, “professional-looking” graphs, and graphic designers should know better, but they prefer to sacrifice data on the altar of Beauty (form is everything, data is a nuisance).

The dominant view among visualization experts (namely Tufte and Few) is that “form follows function“: every ornament in a graph should be eliminated, every object must serve a clear purpose, efficiency should be maximized (labeling series instead of using a legend, for instance). Given the extremely low graphic literacy levels among the general population, this may not always be the best approach.

Misconception #7: A good chart displays the actual values

No. If you label each data point you get a useless table over a useless chart. Labels are not only a distraction but often actually hide patterns in the data. Short labels and annotations can, and should, be used to identify or explain outliers or other interesting data points and circumstances. If your audience expects to see the underlying data then add a link to the table.

Misconception #8: Good Charts Should Be Read at a Glance

No, they don’t. The more complex, the longer it takes. It really doesn’t matter if it takes a second or an hour. What matter is how efficiently the graph  communicates. If a chart takes for ever to be read look for bottlenecks: the series are not easily identifiable, patterns are hidden, demands on the working memory are high, etc.

Misconception #9: The More Detail the Better

What we see as detail can be seen by someone else as clutter. Clutter is the natural child of loss aversion and is is very difficult to remove. If you have 12 competitors your audience will want to see the market share for each of them, even if it doesn’t make any sense. Tufte says “to clarify, add detail”, and yes, 12 competitors in a line chart can be made clear and useful, but you must know how to categorize them and provide a framework to help the user (you can use a large number of categories in a pie chart, for instance).

Misconception #10: It’s All About Selling Your Point, No Nuances

In The three laws of great graphs Seth Godin says that “there is no room for nuance [in a presentation]” and your charts should reflect that. Maybe it is just me, but I hate it when I am not allowed to draw my own conclusions because the data made available by the presenter is too biased towards his/her own points of view. Depending on the situation, a clear path that is supported by a lot of details is much better than a yes/no pie chart.

Misconception #11: You Have to Have Color, Lots of Color

Wrong. Color is a very difficult subject. Large surfaces of primary colors like we often see in presentations should be avoided because they are hard on the eyes and, because everything stands out, nothing stand out. A good option is to use grays for non-data elements like grid lines, and pale colors for color-coding. As a rule of thumb, color should always carry some meaning. Use primary colors to highlight a data point or some other small detail.

Misconception #12: A Single Chart is Enough

It is not. We live in an increasingly complex world, and traditional charts are very simple tools. While we wait for a new set of charts to be invented, we can use interaction (see below) and multiple charts to create a richer picture. That’s why scatter plot matrices, small multiples or trellis displays, and specially those multiple variations of executive dashboards are much more powerful than a simple chart.

Misconception #13: Charts Are Interchangeable

They aren’t. You can use a column chart or a line chart to display a time series, but while a line chart performs better than a column chart when reading trends, it is easier to compare data points using a column chart. Most visualization experts will tell you that you should use a bar chart instead of a pie chart (also because it is easier to compare data points), but a pie chart gives you the perception of a whole that is absent in a bar chart. Every graph has its own strengths, and you should select the one that suits your needs.

Misconception #14: Create It and Forget It

Don’t. Making sense of your data is a process of exploration and discovery. A pattern in a subset may be hidden by a noisy background. Different measures may lead to more complex insights. Creating a chart that the user can interact with should always be your primary goal. Unfortunately, that’s beyond the skills of an intermediate Excel user (if you what to learn about interactive charts my Excel dashboards may be a good starting point).

*

This post lists 14 widespread misconceptions about charts, but probably is a very incomplete list and you may not agree with all of them. What misconceptions would you add/remove?

*

[Update: Jon has been writing extensively about Excel 2003 and Excel 2007 (by the way, it's a great resource that helps us to see through the marketing noise). I said in the comments below that I prefer to use Excel 2007 charts to post images in this blog. He doesn't agree and he tries to prove in his last post that charts in Excel 2003 are actually better. He uses good examples to prove his point but I still believe that this (Excel 2007):

looks better than this (Excel 2003):

Yes, probably there is an "overaggressive anti-aliasing", but the line in Excel 2003 is too "crispy" for my taste. Again, it is just a matter of creating images for a blog, not exactly for serious work...]

Post from: Jorge Camoes' Charts

Well, since we are witnessing the end of the world-as-we-know-it, computer scientists at the University of Utah decided to give a little push, visualization-wise. They are designing a computer application “they hope eventually will allow news reporters and citizens to easily, interactively and visually [analyze] election results, political opinion polls or other surveys”. They boldly state that they “have developed new techniques for exposing complex relationships that are not obvious by usual methods of statistical analysis” (press release). And what are those new techniques? A doughnut chart:

The outer ring labels the series and the inner ring displays the data. Apparently you may add as many series as you wish and you can filter the results by socio-demographic characteristics. There is a video demonstration here [via].

This is the kind of joke that I would expect to be related to April Fool’s Day, but they seem to be serious about it. No one told them that showing part-of-a-whole is one of the few strenghts of circular charts, that when people see 52,7% they see a pie cut in half, not a quarter, that “whole” mean 100%, not 200% or 300%.

Regular readers know that I rarely utter such harsh comments on visualization ideas and applications (I even tried to create a dashboard using Crystal Xcelsius), but this is the stupidest idea of the year. They should know better (here are some tips).

By the way, I found this through a post by Sarah Perez at ReadWriteWeb. She writes: “unfortunately, the poll-analysis software isn’t quite ready for prime time. What a tease!” Fortunately, it is not! And judging from other posts, they could use an information visualization consultant. 

Well, perhaps I’m missing something. Am I?

Post from: Jorge Camoes' Charts

Steve Rubel shares with us how he spent his time online over the last two weeks. He uses the Firefox add-on PageAddict to monitor the time. He writes:

As you can see almost all of my time online is work related. Still I can see that I need to shrink my social network usage a little bit and increase my time with documents, web applications while also keeping RSS contained. I also need to go through the undefined section to see if there are big groups of sites that can be tagged.

I would say that it is hard to see what Steve Rubel wants us to see. It is not his fault, of course, he is just pasting a chart from the application. I do see something interesting: since he spends “the vast majority of [his] computing time” using Firefox (let’s say 80%) he’s using his computer around 4.5 hours a day only. That’s nice…

If I wanted to visually track my time online (I should…) these are some of the options I’d like to have:

• Color-code work/non-work related categories;
• Label the x axis with dates, not “days ago”;
• Remove non-working days;
• Use small-multiples to track each category;
• Use weeks instead of days;
• Annotate outliers;
• Show planned vs. actual time spending;
• Minimize the “undefined” category;

I’m installing the add-on. Hope I can have an interesting dataset to share by the end of October.

Do you use these tools? Do you like their reporting functionalities?

Post from: Jorge Camoes' Charts

Post from: Jorge Camoes' Charts

This article goes much against conventional wisdom about pie charts (and doughnut charts) by answering these two simple questions:

• Can we use a large number of categories in pie charts? (Yes, we can.)
• Can we make a productive use of the apparently useless doughnut chart? (Yes, we can.)

Disclaimer (Sort of…)

Let me start by declaring this: I believe that the analysis of simple proportions is, by its very nature, very limited. It only scratches the surface of the data and it is useless for serious, decision-making processes.

A circular chart is poor because the underlying message is poor. If you can run a business using pie graphs to make sense of your data please let me know what market are you in, because I want to be there too (well, not really…).

Pie chart belong to the media and to some simple presentations. Leave them there. And don’t make the charts you see in the media your role model.

The part-of-whole issue

That said, one must recognize that proportions are so pervasive and hard-wired into our brain that escaping them is almost impossible.

A circular chart conveys perfectly the idea of part-of-whole relationship. You can’t use a bar chart to show this relationship because the whole just isn’t there! Yes, you can use percentage scales, yes you can say it in the title, but it isn’t the same thing, is it?

As I wrote in my previous post on loss aversion, each chart answers a question from a different perspective. Charts are not interchangeable.

Often pie charts are used just because they may look better (this is, of course, in the eyes of the beholder) but what the user really wants/needs to know would be better answered by a bar chart. This is a problem of graphic literacy and information management. It has nothing to do with the intrinsic qualities of pie charts.

The limit of 4 to 6 categories in pie charts

There is a widespread believe that you should not use more than four to six categories in a pie chart.

That’s is wrong or, at the very least, very incomplete.

In fact, you can use as many categories as you want, and still get meaningful insights from the chart. Problem is, you must know what to do with your data (graphic literacy and information management, again), and a large number of bad charts come from this simple fact: people don’t know what to do. Garbage in, garbage out.

“The Secret Strenght of Pies”

Here comes the fun part. In an article published back in 1991 by Ian Spence and Stephan Lewandowsky, titled “Displaying Proportions and Percentages” the authors write:

“the pie chart outperforms the bar chart for complicated comparisons, suggesting that the perceptual addition and comparison of components is inherently easier with the pie chart than the bar chart.” (emphasis added)

(By the way, the authors also say that this advantage will be lost if you “explode” the slices.)

It is not difficult to believe that it is somewhat easier to sum the areas of slices in a pie than it is to imagine the combined heights of bars stacked on one another.(…) Regardless, the fact remains that a comparison of two sets of summed parts is rare in the real world. But, by all means, should you ever need to display data for this purpose, a pie chart would serve you well.

Please note that Stephen Few, in his highly regarded book “Show me the Numbers” says:

“I don’t use pie charts, and I strongly recommend that you abandon them as well.”

Few acknowledges that pie charts “could serve you well” in a very limited set of circumstances (”a comparison of two sets of summed parts is rare in the real world”).

Is it really rare? It may be, but that’s because people don’t know what to do with their data (again). Let’s see.

You have 10 or even 20 categories and you want to use them all (your loss aversion tendency?). Because 20 ungroupable categories are rare in the real world, you should be able to visually group them, using a color (hue) for each group and a different saturation for each category. By doing this, you are adding layers of detail, and the reader will be able to select the level of detail that suits his/her needs. This works best when using an interactive chart because you don’t have to label everything (just use your mouse to identify on-demand the more relevant detail categories) but even a static chart can be used (in this case, label only the relevant details).

The Consumer Expenditure Chart

I used this methodology to design the consumer expenditure chart above, with living expenditure (on the right) and discretionary expenditure (on the left).  As you can see, living expenditure accounts for almost 60% of the total. That’s something you can’t easily see with a bar chart.

Then, there is a second level of detail, where you have categories like Housing (more than half of living expenditure) or Transportation. And finally, you could use your mouse to identify those detailed categories in the outer gray ring.

I’ve added some arcs to compare the profile of total consumer units to consumer units with five or more persons. Each arc always starts at the same degree of the corresponding slice. Different proportions lead to gaps or overlaps. Please note that this is not a core feature of this chart. Just wanted to play a little with comparisons (an obvious issue: since the first arcs are closer to the center, a gap between them is different than a gap between the last arcs).

The Secret Strength of Doughnut Charts

As we saw above, pie charts are better than bar charts when comparing proportions. But, as soon as you add a second pie chart you are trying to compare proportion A1 with proportion A2, not proportions A and B of the same pie. There is a shift in the analysis and the pies become useless (use bar charts instead).

Just because you can merge both pie charts in a single doughnut chart it doesn’t mean that you gain efficiency, because the essential problems remain in place.

For many, a doughnut chart is a bad mutation of a bad chart. But if, just if, two bad’s become on good? Could a doughnut, if correctly use, become a kind of pie chart on steroids?

Let me emphasize this: never use a doughnut chart to compare series. I don’t, and I strongly recommend that you should avoid it as well… Always use a doughnut chart to add detail to a series. That’s the secret strength of doughnut charts.

And please, please, could someone write an article on doughnut charts for the English Wikipedia?

I made this chart in Excel

In case you are wondering, you can make the Consumer Expenditure chart in Excel, 2003 or 2007. Instead of the default theme colors, I used some of the colors that will be available in Chart Tamer (thanks, Andreas!).

Conclusion

Pie charts do not deserve their bad reputation. They seem to be more efficient than bar charts in some very specific tasks, like  comparing combined proportions. We should take advantage of that by adding multiple levels of detail. We shouldn’t be afraid of using a large number of categories, provided that those levels of detail are clear and meaningful.

The doughnut chart is the most misunderstood of our chart toolbox. It is seen as completely useless because two series should not be compared using circular charts, but that’s not what doughnut charts should be used for. They should be used to extend the power of pie charts, managing efficiently the level of detail that we need to add to create more insightful charts.

Is this a good way to use pie and doughnut charts? Please share your thought in the comments.

Post from: Jorge Camoes' Charts

JunkCharts writes an interesting post on how loss aversion can happen in chart-making. The general concept of loss aversion tells us that “people strongly prefer avoiding losses than acquiring gains”. Translated to chart-making, it means that there is a “tendency to avoid losing data at any cost”.

“To clarify, add detail” says Tufte. Corollary: you should make data-dense charts and maximize the data-ink ratio. Problem is, this fits too well into the loss aversion tendency. Take the above chart, for instance: does it make any sense to add those nine series to a single chart? What insight do you get from it? Only one: the designer don’t know how to handle a larger number of data series.

Remove irrelevant data series and you risk a mutiny on the Bounty, even if relevant trends are easier to detect. It is absurd, but very human.

So, how can you give the users all the data they expect while keeping the chart clean and readable? Well, to clarify, add detail to existing patterns (that’s what I just did to Tufte’s sentence…).

Tufte talks about “data layers”; Ben Schneiderman’s Visual Information-Seeking Mantra (”overview first, zoom and filter, then details-on-demand”); the focus+context technique. All they convey a simple idea: prioritize your data. Know what is relevant and what is nice to have. Don’t give the user a final product. Make an interactive chart and let her discover what’s inside.

I see this loss aversion tendency at work every day at the office. Do you too? How do you handle it?

Post from: Jorge Camoes' Charts

Actually, I am more in the mood to defy current “crowd wisdom” about pie chats.

Search the web for “pie chart” and you’ll get more than one million results, and a depressing picture of human knowledge. Browse the first 100 and what do you get? Some educational(?) sites (poor kids), tutorials (Excel, php, java, Illustrator), humor (here, here, here), bad (here, here, here, here, here or here) or just plain stupid examples. You’ll also find them in in court or fighting government (who could ever imagine that?). I’ll leave for another post what the Wikipedia and the pie chart thread in Tufte’s Ask E.T. say about pie charts (Stephen Few’s Save the Pies for Dessert is not listed within the first 100 results).

An old litany

Some of these sites discuss the use of pie graphs, but they usually recite the same old litany: our perception is bad at judging angles, you should use no more than five or six categories, don’t use them to compare series, Cleveland’s findings, etc. (there also is at least one unfair comparison between pie and bar graphs and one very aggressive rant against them).

If there is something that I would like to have written about pie graphs it is this Expert notes at ManyEyes:

Pie charts have a mixed reputation. They are popular in business and the media but many information designers have criticized the technique. Some claim that the pie slice shape communicates numbers less exactly than other possibilities such as line length. But this remains unclear in the context of proportions: for example, we have seen no studies that looked at the task of judging whether an item is more or less than 50%. It’s also unclear whether exact communication of numeric values is the only evaluation criterion; at least one study indicates that use of a pie chart for analyzing a problem as opposed to a bar chart changes the way people think about the problem.

This is clearly more constructive than saying that “they are as professional as a pair of assless chaps” (less funny though).

Not all charts are born equal

Current wisdom presumes that bar graphs and pie graphs are equivalent. For that reason, bar graphs should be used, always. After all, they are more efficient, right? But if they are not equivalent, as the above quote suggests? Take a time series, for example. If you want to see trends, you’ll choose a line graph; if you want to compare data points you’ll use a column graph. They are very similar, but by choosing one or the other, the designer is making a choice of how he/she’ll  look at the problem. Bar graphs and pie graphs are very different, so shouldn’t we think twice before selecting a bar graph because of its presumed superior efficiency?

This disdain for pie charts has its roots in Cleveland’s work and in Tufte’s and Few’s writings. Their positivist view towards information visualization may be as relevant as the classic economic theory and its presumption that consumer always take the rational decision, but are we not all predictably irrational? I agree with Robert at EagerEyes when he says:

There is no doubt that we need to be careful about the choice of visual representation, and that we need to encourage the use of good charts and criticize the bad ones. But that doesn’t mean we can get lazy and squeeze everything into a few standard charts types we’ve been using for decades. That is especially true if we want people to actually care about what we’re trying to show – and not bore them to tears.

We should probably try to be more rational and circumspect in a decision-making environment and do not use the media as our role model, otherwise business visualization may become useless. However, ruling pie charts out is not the wisest decision.

Simple rules are made for beginners. Let’s break some. How about this one:  “you should use no more than five or six categories in a pie chart”. Are you sure?

(Before that, we must re-read what Cleveland said and what others said about Cleveland. That’s the next post.)

Post from: Jorge Camoes' Charts

2th Place
(also nice, but too many slices, and I don’t like the title around the pie)

Lateral Pie-Thinking Award
(well, perhaps someone just messed up the template)

Designer’s Pie Charts Award
(data? what data?)

Seth Godin’s Pie Chart Award
(”makes an obvious point, no nuances“)

Consensus Pie Charts: The Venn Pie

Consensus Pie Charts: The Line Pie

Consensus Pie Charts: The Bar Pie

Flash Gordon Pie

We Try Harder Award

Post from: Jorge Camoes' Charts

Now, let’s see what the future brings.

Post from: Jorge Camoes' Charts