|
|
Post by sciutoalex on Apr 11, 2016 3:56:24 GMT
(Alex + Rushil)
Authors
Bill Cleveland - PhD in statistics from Yale in the 1960s. Afterwards worked at Bell Labs where he was probably highly influenced by John Tukey's work in exploratory data analysis. Outside of statistics, Cleveland is best known for his work measuring how people perceive graphs.
Robert McGill - We could not find biographical information about him, but looking at Google Scholar results, he wrote a number of papers with Cleveland, Tukey, and others measuring the accuracy of graphical perception.
Summary
So just how good is your "eye-brain system" at extracting quantitative information from graphs and charts? Cleveland and McGill seek to find out using rigorous methods to measure perception. Before this paper, much had been written on the craft of constructing charts and graphs, but no one had established what "perceptual tasks" resulted in the most accurate judgements of visual encodings of quantitative data.
Reviewing prior literature on human perception the authors hypothesized an order of perceptual tasks based on accuracy:
1. (most accurate) position along common scale
2. positions along non-aligned scales
3. Length, direction, and angle (unclear what internal ordering would be)
4. Area
5. Volume, curvature
6. (least accurate) Shading, color saturation
They performed a number of experiments on 50 people using common graphical forms (sample distribution plots, bar charts, pie charts, divided bar charts, shaded maps, curve-difference charts, cartesian graphs, triple scatterplots, volume charts, juxtaposed cartesian charts) and they mostly confirmed their hypotheses.
Discussion
Viewing complicated encoded images requires more than just Cleveland and McGill's perceptual tasks. What other cognitive tasks must viewers perform when looking at a graph? How does attentional selectivity and identification work when you view a graph?
When creating graphs and interfaces for people, do we always want to use the most accurate encoding? In what circumstances is less accuracy desirable? Why?
This article was written in 1983. Charts and graphs were specialized and rarely used. Has that changed in the 33 years since? Do people today have a greater amount of perceptual work to do than people in 1983 did? Are we more graphically literate?
|
|
nhahn
New Member
Posts: 22 
|
Post by nhahn on Apr 11, 2016 6:12:40 GMT
Regarding the question about using the most accurate encoding versus less accurate encoding, I feel that this heavily relates to a story a person is trying to tell with a particular graphic. I think the article tries to somewhat avoid this, by instead discussing it in the terms of "if you want to show X you should probably try and use these tools". Graphics (as we have probably seen many times) can be used somewhat unethically to show a difference where there really is known. By using techniques like scale adjustment, you can try and convince individuals that there is something significant about a situation, when there really isn't. I think that less accurate encodings might lead to possible misinterpretations of the data, but at the same time can help to point out something interesting.
Aside from that, I think there is the question of using graphics to perform exploratory data analysis. Since a graphic can skew your understanding of data, how can you, when exploring a dataset using a graphical tool, ensure that you are being exposed to the right features? This might be a problem with other non-graphical tools, and in the end come down to expertise. I think this question relates heavily to the Kinetica paper as well.
|
|
|
|
Post by stdang on Apr 11, 2016 20:08:26 GMT
I agree with Nathan in his discussion of understanding what you would like to encode in your representation is most important in creating your visualization. Accuracy is entirely dependent on what dimension you are emphasizing is accurate. Just looking at the various map projections available, you can find that different values are placed on the projection design that leads to the distortion of specific dimensions of accuracy to emphasize the perception of specific features of the globe. In fact, looking at various map projections, you can see the applicability of Cleveland & Mcgill's work. Looking at how area and curvature influence perceptions of country scale as well as how lines connecting cities may differ in the geographic regions they overlap thus convey different senses of distance and trajectory.
I woudl also disagree that charts and graphs were rarely used. While more professions may be data driven now, a lot of the usage of visualizations are not exclusively found in the professional environments. Advertising, marketing, and politics has long been leveraging visualizations and information distortion for various ends and to the extent that this has changed since the time of this paper, I'm not too convinced the world is that different.
|
|
|
|
Post by JoselynMcD on Apr 11, 2016 21:30:15 GMT
RE: This article was written in 1983. Charts and graphs were specialized and rarely used. Has that changed in the 33 years since? Do people today have a greater amount of perceptual work to do than people in 1983 did? Are we more graphically literate?
While I think that people are exposed to more charts and graphs these days due to the fervor over data visualizations, I don't perceive (and studies seem to suggest) that the general populations has come a long way in terms of our ability to 1. suss out the core concept (if there is one) off the the chart/graph 2. discern when we're being manipulated or 3. critique a graph or chart like we do many other types of media that attempt to persuade. I think people have the same amount of perceptual work to do now, but more people are being asked to do it than ever before, but without the tools to do so. As a lover of data and visualizations of data, I'm personally excited by the growing access and interest, but I fear that there is a false perception that just because something is in chart/graph form that it is inherently valid communication of data/information. In my humble opinion, designers in this space should be dedicating resources to make sure the way the data is being communicated ethically. As recipients of data in charts and graphs, we need to be wary about what the chart is communicating and consider the choices made in it's construction.
On a less dire note, I'm just going to put it out there: I think every single one of the charts in this study were hideous. Perhaps we should be talking a bit more about aesthetic and appeal when looking at design implications for communication designs.
|
|
|
|
Post by Anna on Apr 11, 2016 22:22:16 GMT
Continuing with this conversation-- I'll go as far to say that in general, I don't trust graphs. If I want to get an understanding of what an author (graph maker) that I already trust is positing, then a graph is useful because it can sometimes give me a quick understanding (though of course, so could succinct text). If I really want to evaluate an empirical argument, I'm going to want to look at raw numbers and statistical measures.
And yes, I agree with Joselyn that these graphs are pretty darn ugly. But at the same time, I wonder what is really the value of making very aesthetically pleasing graphs? Does this have the potential to further occlude the 'truth,' in that people may be more willing to trust more aesthetically pleasing graphs?
Ultimately, data visualizations don't really mean anything to me. I generally enjoy making them, mostly because it seems that other people like them and you can challenge yourself to make them look pretty or interesting, and I like looking at pretty/interesting ones and being impressed by the skill of the designer. But (I think) I'd generally rather read things than look at a graph. I'm trying to decide if this is true. I think it is.
|
|
Qian
New Member
Posts: 20 
|
Post by Qian on Apr 11, 2016 22:25:28 GMT
I very much agree with Joselyn on "designers in this space should be dedicating resources to make sure the way the data is being communicated ethically." Most of the conclusions this paper presented are not surprising (i.e. length is easier to compare than area, etc) but its applied effect really depends on toward which direction designers try to steer. Continuing on Nathan's comment about "accurate encoding versus inaccurate". I wonder if encoding accuracy is equivalent to cognitive accuracy. One of my favorite papers in clinical risk communication is titled "Why a 6% risk of cancer does not always feel like 6%" -- Even people precisely perceive a graph as "6%", its meaning (risk perception) largely depends on what it's compared to rather than the number itself. But then how can we communicate the exact "6%"? This article was written in 1983. Charts and graphs were specialized and rarely used. Has that changed in the 33 years since? Do people today have a greater amount of perceptual work to do than people in 1983 did? Big data has came and data viz research seems closely surrounds that. Summarization is prone to biases but becoming necessary. Many visualizations overview distributions or so with color/ area and also enable users to zoom into accurately encoded details. This seems a huge step from the singular graphical perception study of this paper. |
|
|
|
Post by jseering on Apr 12, 2016 0:19:28 GMT
I generally agree with what everybody has said about ethics of graphics, so I'll do my best to play devil's advocate (though it'll be hard). Is it fair to not trust graphs? We certainly know that data can be falsified or chosen selectively; axes can be changed; labels can mislead, etc etc, but all of this is certainly not unique to graphs. We (should) know that it's important to read any source critically, whether it's a graphical source or a text source or an audio source. Are graphs really somehow inherently more misleading? What is the distinction between a misleading graph and a misleading statement of the same information?
|
|
|
|
Post by francesx on Apr 12, 2016 1:14:49 GMT
RE: This article was written in 1983. Charts and graphs were specialized and rarely used. Has that changed in the 33 years since? Do people today have a greater amount of perceptual work to do than people in 1983 did? Are we more graphically literate?
I agree with Joselyn regarding her remark that the graphs in this paper are a little bit hideous. I understand that the paper was published in 1983, but these guys need to read some of Edward Tufte's work.
What I see as a change in the 33 years since this paper, is that people have a similar amount of perceptual work to do as the people in 1983. We might be more "used to" and "familiar" with certain charts and graphs, but with the advance of the technology, more forms of visualizations are possible. To begin with, we are surrounded by more colors and hues, but also 3D graphs, as well as other types of charts or infographics. I would also be skeptical to think that people's abilities to work with 2D and 3D objects in 2D space has evolved much in 33 years.
As a side note, for all my fellow PhD students; if you ever find yourself doing research on data visualization or you are trying to create a visualization of some data for a paper or presentation, I highly suggest reading some of Stephen Few's books (check out also Cole Nussbaumer Knaflic).
|
|
toby
New Member
Posts: 21
|
Post by toby on Apr 12, 2016 1:52:33 GMT
+1 on what Joselyn said. I too think that though general public are much more often exposed to graphs and charts, they are not really significantly more "graphically literate" in terms of deriving the core concept, discern the manipulation or critique a chart or graph. For the paper content, I would really like to see a similar analysis on human's perception of newer data visualization techniques (e.g. Kinetica), like the animations, interactive data visualizations etc.
|
|
judy
New Member
Posts: 22
|
Post by judy on Apr 12, 2016 2:14:51 GMT
I agree with Qian: Big data and the data visualization that has become such an important part of how we understand big data, seem to be a different beast entirely than the graphs presented in this paper.
Besides the graphical features of a data viz, I think we see an increase in using visualizations as journalism--not supplementing an article, but replacing the article. Therefore, there is increase pressure on the designer to express a POV or a whole story with a single interactive graph. This is a subjective process (not that analysis could ever be wholly objective).
I wonder if I would rank the perceptual tasks in the same order for an interactive visualization. For example, it seems to me that color usually represents high-level information (e.g. Republicans vs Democrats) that I might perceive first, especially when there may be many "points" at many "positions."
|
|
|
|
Post by cgleason on Apr 12, 2016 2:43:57 GMT
Re: This article was written in 1983. Charts and graphs were specialized and rarely used. Has that changed in the 33 years since? Do people today have a greater amount of perceptual work to do than people in 1983 did?
I think that most people gain familiarity with specific types of graphs. We are very exposed to bar charts, so the majority of us can read and understand them. Sure there is a perceptual component, but most of that can be overcome with training. What's changed since 1983? Honestly, not much. People are still exposed to pie, line, and bar charts. Despite the influx of data visualizations, I don't think any other chart has actually made it to mainstream, partially because none of the other graphs have been useful enough. It might be a perceptual problem, but how often do you really need to use a stream graph?
Re: When creating graphs and interfaces for people, do we always want to use the most accurate encoding? In what circumstances is less accuracy desirable? Why?
I think a graph is as much a medium for communication as your text. Just as you would choose a less accurate form of explaining your study off the cuff compared to your paper, there are places to use less accurate graphs. Namely, when trying to get across a broad point but punctuated with a dab of data. The only caution here, of course, is that if you have an inaccurate graph and get called on it, it can be harder to live down than an inaccurate paragraph. Graphs seem to convey some air of authority, and misrepresenting them is more fraud-like. Nevertheless, they will always be biased and should be leveraged to get your point across in whatever (ethical) way you feel.
|
|
|
|
Post by bttaylor on Apr 12, 2016 3:07:32 GMT
What I found most striking was that the conclusions of the paper seem to have made very little impact of the types of graphs that are created (and the data visualization packages that are used to create them). I'm curious why stacked bars and pie charts are still so common when it's been shown that the dot charts more accurately convey the information. Is this just a matter of habit? Or, as Joselyn et al. noted, do we just find the aesthetic of a pie chart to be worth keeping it around?
I'd also be curious to see how modern data visualization relates to these 'elementary perceptual tasks'. Does being able to interact and manipulate data change the accuracy of perceptions? Does the fact that you can bring up the raw number by hovering over a point make this a moot point? The author mentioned the superiority of tables for conveying accuracy, and for interactive data we certainly can have the best of both worlds.
|
|
|
|
Post by fannie on Apr 12, 2016 3:20:29 GMT
Re: When creating graphs and interfaces for people, do we always want to use the most accurate encoding? In what circumstances is less accuracy desirable? Why?
I'm comparing graph visualizations to other kinds in my research, and one thing that have came up was whether or not the information was more up for interpretation by viewers. I'm dealing with data with meaning that's not very black-and-white, so having a graph allowed us to be accurate in just presenting the raw data, allowing for anyone to interpret as they wished, so I think it can still be accurate in the sense of the data but misinterpreted. On the other hand presenting data that was inferred was questioned more in that the way we presented it was still powered by all the same data but was presented in a way that did not match users' perceptions... but the meaning might be more consistently interpreted. For when to use them, maybe less accuracy could be useful for a general feel of something (like being "close enough" or "around there").. generally when designing how we present data we should probably ask what sufficient accuracy is.
|
|
|
|
Post by julian on Apr 12, 2016 4:06:27 GMT
This article was written in 1983. Charts and graphs were specialized and rarely used. Has that changed in the 33 years since? Do people today have a greater amount of perceptual work to do than people in 1983 did? Are we more graphically literate?
Like Steven I think graphs have not changed radically, if anything some aspects of it have become dynamic but the most commonly used graphs are still the same. Of course there are many new ways to visualize data that were not even possible in 1983 however, none of these new visualizations are standard yet. @anna: I think it is good to not trust graphs but it is usually very easy to tell if someone is trying to oversell something just by looking at the scale for example. I think the value of many visualizations really comes from unexpected results and unfortunately, many statistical tests or statistics make very strong assumptions of the data that usually hold only for very large data sets (asymptotic theory). In HCI data sets can be small and do break the IID assumption in so many ways, this is of course not unique of HCI but pretty much any field working with human generated data. Hence, looking at only statistics is actually dangerous, the best is usually to look to both stats and graphs. In response to @nathan how can you, when exploring a dataset using a graphical tool, ensure that you are being exposed to the right features?: You need many different visualizations. However this is only possible for small data sets. When the number of features is too large, familiarization with the data set and creating a testable hypothesis about the problem is the way to go. Methods for finding the features that encode the most information exists (Mutual information) and there are multiple transformations that can reduce the dimensionality of the problem by rotating the basis of the data into one that maximizes variance (PCA). However, none of this comes with the guarantee of selecting the "right features" (this is also highly dependent on the problem). Also in many problems there are not right features and all of the features are important (i.e., they all contribute to the solution of the problem). Take for instance a model for predicting if a person is going to click on a given Ad. Google's model (as stated by one of the engineers working on this) has over a billion variables I doubt they try to visualize this or even pick the right amount of features. One could say they do have the computing power for simply doing this, basically they may not even care, however, that is not the case, they do have the computing power but computing this model takes one entire day. This is really nothing for the kind of problem they are solving, however it probably costs them to not being able to update this model more frequently. Also they seem to have put a lot of work on computing this model in just one day. One more interesting thing, the model they use is actually logistic regression  .
|
|
|
|
Post by judithodili on Apr 12, 2016 5:46:05 GMT
@brandon - I think graphical representations like the bar and pie charts are preferred because it takes very little skill to interpret them. Usually, when representing data visually, the point is convey the main idea behind it vs focusing on people understanding it in very accurate detail.
RE: When creating graphs and interfaces for people, do we always want to use the most accurate encoding? In what circumstances is less accuracy desirable? Why?
In my experience - no! If I put up a very accurate graphical representation, and my audience cannot look at it and comprehend it immediately, than IMO the point of using a graphical representation is defeated. Less accuracy is not necessary undesirable, however, comprehension (depending on your crowd) is key. The conversation of ethics of graphics is interesting - what is the line when trying to maximize comprehension? In my experience, as long as the representation doesn't falsely represent the data in any way then I'm good to go, but I can definitely see how the the waters can get muddy.
|
|
?
.