Costs and Benefits of Structured Information Foraging

I think because of the overwhelming dimensions, the simple way of structuring and categorizing information might not be effective. Here, visualization techniques can play a significant role. For example, for a lot of electronics shopping websites, we can directly add multiple objects for side-by-side comparison. Also for Amazon, the dimensions are reduced to more user facing such as pricing, user satisfaction, etc. These visualizations can provide users with more direct feedback. I agree with the others about the task being not realistic and complicated enough, and I think the tool can be another factor.

One question I have about the measures is how informed and confident are the users to their decisions. Since they are not actually making their decisions on buying a product, it might only be a sense making tool rather than decision making. And I think there's a gap between the two.

I am surprised every time I start a project...research project, group project, lit review, design...how strong the urge is to create structure from the get go, and how every single time I do, I end up throwing out the whole structure and starting again in a less-structured way. Why is that urge to structure so strong? Is it just wanting to get an answer and be done with it? Is it out of some sense of self-importance or genius, that I think a structure can spring from my mind without first gathering/brainstorming? Is it because I'd feel lost without one?

Recently, I've also been wondering if there's a value to giving into that urge to create structure from the beginning--if I also remember not to be attached to that structure. Last year a group of us in PNT played around with the idea of "prototyping the search," creating a draft of a model early on. The idea came up again a couple of weeks ago when a group of researchers were shaking their head at the horrible waste of time their early work had been. But is it really a waste of time? Perhaps for complicated tasks, we need some structure not only to get us started, but to give ourselves something to respond to, to shape, to poke holes in.

This is an interesting paper. Intuitively, it does make sense that structure would come towards the end to be more useful. Until a person has seen a pattern, it is hard to draw out a conclusion. This somewhat aligns with the other reading where we can pick out differences in similar things easily. Once we start seeing similar things, it is easy to set out a structure by noting the differences, and hence it makes sense that the having a structure towards the end was productive.

I like Ana's point about tasks which explicitly don't require building a mental model. If we are dealing with a game of imperfect information, then building a structure/mental model is detrimental. This is tangential, but I would be curious to know what are the chaotic alternatives for situations with imperfect information.

I'm really curious how something like this would apply to other information domains. While there is discussion of information foraging being applicable to many domains (patient diagnosis, research literature) this tool seems really structured towards making within category comparisons. While I think that is useful, I don't know that it's quite as wide-reaching as I expected from the intro discussion (which frame it as tool for capturing information).

I think a lot of the discussion of whether delaying the structuring has additional costs is probably tied up to this type of comparative domain. I could very well imagine a similar study with apartment hunting where the mental model about what is important is changed by what features are available. However, taking Anthony's suggestion about planning a trip, which doesn't really have this zero-sum comparison, might result in a very different result.

It seems like there are probably a lot of different sense-making mental processes that are going on and it might be that a tool appropriate for one just won't work that well in another context.

@michael rivera: I think the task was just right for the hypothesis it was set to test. Remember that this is about sense making on a sensible amount of time not over the years or months which is what will probably require to understand string theory. 

@mary Beth: I understand your concern with no comparing against a base case however, I believe the vision of the paper is to be able to capture and share (over the internet) this sense making process, hence a tool is definitely necessary to be able to scale up to millions.  

Conditions 2 and 3 are interesting and they make sense with out previous readings, as a concept is forming initial exemplars or even properties of the exemplar become part of the mental model, as the user learns more this model gets more refined so, for C3 and C2 the models are better. In a way, maybe dissimilar stuff later makes more sense when a core structure or pattern is recognized.

It is interesting that the car seat and camera times do not follow the same pattern for the different conditions. This probably reveals some mediation effect between the complexity(time) and the sense making topic. 

One of my big issues here is that conclusions are being made about the structure of participants' mental model of the information space, when what is being evaluated is a linear list of dimensions along which individual items are being evaluated, rather than a network-based approach, with some semantic information encoded in the edges [1, 2]. As we read in the concepts chapter, a feature list is not the most robust representation of a single concept, much less the conceptual structure and inter-relationships of elements of a mental model of an information space. Further, mental models are inherently internal, fragmentary, and dynamic, so to ask whether we need mental models is really asking whether we need externally-represented versions of them.

I agree with Mike R. (et al) that for a simple information-seeking and decision-making support task such as searching for a camera to evaluate and purchase, this would be useful (because of the simplicity of a linear list of feature dimensions rather than the conceptual structure of a mental model). But, for open-ended sense-making tasks such as exploring a new research domain, the conceptual structure would need to be more complex and easily revisable to reflect the dynamic nature of mental models. Now, Kittur et. al did incorporate revision of their list of dimensions, so this seems to address people's concerns about "getting locked into" a mental model representation with such a tool. If a more richly structured conceptual network were implemented to represent the mental model, similarly allowing users to revise its structure as their mental model's structure changed seems like it would be beneficial.

Finally, I also agree that time seems like not the best metric here. Are we really trying to make the sense-making process "more efficient"? For researchers, this process will continue our whole lives. Why would we want to speed it up, if that's at the cost of the richness of our representations? Perhaps some other metric would be more useful here, like "new insights gained" (the "unexpected phenomena and illuminating insights" of the Malone paper, or something).

Finally (pt. 2), for our P&T project, we automatically generated such a network structure, though of papers for a lit search, and not of the concepts as the nodes. We also found that efficiency or accuracy metrics (number of papers found, number of "important papers" found) were not as interesting as the perceptions of the users such as feeling like they were getting more done, or understood the information space better, or had some new thought about their search process or the domain. However, we also did not allow for the user to change the structure of the network after it was generated, an area we wanted to explore more in the future.





J. Langan-Fox, S. Code, and K. Langfield-Smith. Team mental models: Techniques, methods, and analytic approaches. Human Factors: The Journal of the Human Factors and Ergonomics Society, 42(2):242–271, 2000.
D. Gentner. Structure-mapping: A theoretical framework for analogy.Cognitive science, 7(2):155–170, 1983