|
|
Post by stdang on Apr 5, 2016 3:22:57 GMT
I actually think this was a very clever way to establish these results given the context. During ideation, it is pretty well established that expertise supports creativity. Thus for more engineering or advanced scientific tasks, an average MTurk worker is not likely to have particularly creative outputs. However, for everyday tasks such as sleeping on a bus or plane, MTurk workers are much more likely to have sufficient experience in these spaces to be considered proficient if not somewhat expert at that particular task. Then using example products that were well within their realm of experience likely contributed to the success of the study. Making this task more focused might change the results depending on the task you have them solve. If it is still an ideational task, but you are attempting to constrain their thinking to a smaller subproblem, then it is possible that you are depriving them of their "expertise" and they are not able to ideate from a specific perspective due to low familiarity. The open-ended nature of the ideation task as expressed, allows the participants to draw from their own specfic perspectives with the notion that enough participants will provide a variety of perspectives on the problem at scale.If you were to change the task to problem solving or categorization or decision making, then this is well outside the claims of the paper as these recruit different cognitive processes.
|
|
aato
New Member
Posts: 16 
|
Post by aato on Apr 5, 2016 3:30:01 GMT
...3) How could we augment an individual’s process by strategically interjecting crowd intelligence?Probably by showing the ideas and schemas generated from the crowd. I would wonder though how this would affect the individual depending on its own creativity, for example would a highly creative individual will benefit from this more or less than a not so creative individual. ... I think this sort of conflict is what has inspired further research in this area. I think people who are pro-crowd intelligence would argue that by augmenting an individual's process, you're not stifling creativity, but getting them past all of the boring and standard ideas and jumping them ahead to a place where they can really apply their creative thought to a variety of already complex and interesting schemas. For me I think scaffolding my creative tasks has unlocked my creativity simply because the act of being creative is so intimidating. Last semester Jess gave me game design assignments. After one week of being her student, had she said "make a game in a week" I think I would have fully panicked and shut down. Instead she started me out with, "come up with 20 versions of the game tag." Sure, I didn't think about every possible game out there, but I got moving a lot faster with constraints. And from there she slowly gave me increasingly open-ended tasks. So maybe the key is to start augmenting a process, but slowly wean a person off of the augmentation. |
|
|
|
Post by Adam S. on Apr 5, 2016 3:39:30 GMT
1) What do you think about the generalizability of the results? This paper takes the theories we read about in the other journal articles and attempts to scale them. How well do you think the authors succeeded?
In agreement with mostly everyone above, I also find the results of these experiments to be generalizable for problems in which there is no definitive right or wrong solution (e.g., creative problems) such that ideation becomes necessary. With regard to scale, the process for distributed analogical idea generation definitely is designed to scale due to its distributed nature. By breaking down tasks into smaller pieces of work and allowing for multiple people with different experience levels to contribute at the different stages of the process helps to lower the barrier to entry and add more diversity in the ideation of solutions.
My biggest gripe with this paper is that the authors mention that, for example, using schemas result in "better" ideas than using examples, yet there isn't a clear explanation or operationalization of what "better" means beyond being more novel and practical. Are these really the two best measures for whether an idea is "better"? Are there alternatives?
3) How could we augment an individual’s process by strategically interjecting crowd intelligence?
Related to the previous question about generalizability, I think this overall concept of schemas can be applied to other domains beyond collective product innovation and design. For example, consider how this could be applied in an education setting with mentorship. The distributed analogical thinking could be applied to helping mentors scaffold learners' behavior to become more self-regulated, for example. Perhaps a system could help facilitate mentors' interactions with learners by providing various "good" schemas for how to communicate effectively with their proteges to achieve common mentorship tasks.
|
|
|
|
Post by xuwang on Apr 5, 2016 4:05:29 GMT
Question 1:
I also agree with previous posts that the results are generalizable to similar tasks, but I have a similar question as Adam, because the authors haven’t defined what means better and more innovative ideas. I think for the groups where participants are provided with examples, it’s possible that their brainstorming will be constrained by the given examples, so the ideas generated may not look as novel, but they can as well be good ideas. And I would also assume if participants are given something to build upon they will probably come up with more reasonable ideas?
1a) I think if participants are told to solve a specific given problem, it will connect to the worked example theory in learning science, which says when students are given worked-out problems, they learn better than when they’re given new problems to solve, because learning worked-out problems will reduce their cognitive load, but students will need to reason about each step of the worked out problem, and abstract from them. I think that also depends on how we’re measuring the outcome, if we use a near transfer problem, probably examples will work better, if we use a far transfer problem, schema may work better.
1b) I think on the one hand we could assume for more skilled workers, examples would work the same, because skilled workers will be better at abstracting schema from the given examples. But on the other hand, we could also argue for people who have more complicated mental models, schema will work better, because more detailed examples could interfere with their existing mental models. (similar to the idea of overscripting)
|
|
|
|
Post by fannie on Apr 5, 2016 4:08:32 GMT
For #1a+b, I think that even if they were skilled, if they’re from different domains the schema needs to make sense to people from all of the different domains (maybe it needs to go through the up-goer). Also, maybe multiple good schemas for a focused problem rather than one good one would help with coming up with different examples. I think as long as these schemas are understandable to a wide audience, I don’t think that being skilled would necessarily mean that better ideas would be generated. I can recall times when I’ve talked about a problem with people unskilled in that domain, but those people would still be able to come up with solutions as long as they got the gist of the issue.
For #3, if we were aiming for smarter/more creative individuals, this could maybe be achieved by exposing them to these diverse schemas or examples so that they can have more in their toolkit to apply to problems. This is similar to what others have mentioned about being exposed to different perspectives.
|
|
|
|
Post by rushil on Apr 5, 2016 5:00:20 GMT
One of the questions that come first to my mind is, "who prunes the list of good and bad ideas?" and "how big of a crowd is too big?"
There is definitely merit in the approach being presented here. As someone already mentioned, idea generation via crowdsourcing has gained a lot of traction and the paper provides a good way to structure that process. However, thinking beyond research and putting it out in quantitative measures, I don't think we are at a stage where this process, no matter how structured can actually be useful in the short-term.
Have they figured out how to simplify and structure the ideation phase of product dev.? Yes, absolutely. But, there is still a gap of how do you do resource management to achieve the same effect with the simplified structure without blowing up your resources.
|
|
|
|
Post by Anna on Apr 5, 2016 10:15:46 GMT
Regarding whether this would actually help the ideation process-- one thing I was wondering about this work is how do we access/leverage analogies from other fields or situations that we don't already know or understand? For example, many of you probably saw Linning's job talk earlier this year, in which she discussed how she used analogies from the natural world to shape technological design. There might be analogies from other fields-- medicine, music, etc.-- that come from obscure/esoteric niches within these fields that would be really useful and applicable for novel idea generation in HCI. Overall, HCI is a lot more interdisciplinary than a lot of other academic fields, and I think this is one of our greatest strengths-- people come in with all these different backgrounds and perspectives that can inspire creative, novel solutions. But we might still be missing out on a lot of potential analogies from less known fields, or strange little pockets of well known fields, that could inspire/shape our work. So how can we go about accessing/understanding/leveraging analogies that build off concepts/entities that we would never think of or even readily understand?
|
|
Qian
New Member
Posts: 20
|
Post by Qian on Apr 5, 2016 13:11:16 GMT
I have the exact same wondering as rushil’s: who prunes the list of good and bad ideas? It seems a constant struggle in such genre of crowd intelligence studies. If judged by experts, the value of crowd intelligence seems more or less belittled; If not, dissemination of its artifact seems difficult. It seems to me a paradox between novice crowd and a “more effective and less reliant on chance” idea generation. In response to nhahn’s question about what IDEO does with a bunch of experts. More often base-domain experts in collaboration with novices (or experts from outside domains) to generate analogies and to design. Which outside domain to choose is predominant to successful collaboration. This pairing eliminates one end of uncertainties, which seems more efficient. |
|
judy
New Member
Posts: 22
|
Post by judy on Apr 5, 2016 13:15:39 GMT
I buy that schema presentation resulted in better ideas. I buy that schemas are easier to apply than examples (although see a huge value in the creative herself developing a schema from the example). I also buy that an ideation task can be broken into steps that a novice crowd can use. But I still can't really let go of "intention" in creativity. Without understanding the intention, or let's say the "problem" that is being solved, it's hard to evaluate the usefulness or uniqueness of even the aesthetics of the solution. So, I could see myself looking through these crowd-generated ideas and saying to myself, "that is genius! 'X' would be perfect for 'Y'!" It's the intention behind my application that makes it unique. Otherwise, it is an exercise, a drill. Artists of all types engage in exercises to isolate and practice certain skills, sometimes something beautiful comes from that drill. And certainly, the more often that you practice it, the more likely you are to generate something beautiful. But there is a next step of taking that doodle created in a drawing exercise and contextualizing (perhaps polishing it) as art. I wonder if the crowd workers felt like they were doing something creative, something fun, something innovative.
|
|
|
|
Post by mmadaio on Apr 5, 2016 13:42:20 GMT
I'm with Judy... I can see this being very useful for ideation, but unlike some others, I really don't see this being generalizable to "problems for which there's no definitive right or wrong answer" (Adam). I think the nature of the well-structured, tightly constrained problem/solution space of inventions or product design makes the ideation process easier. If these were wicked complex problems, or even solvable social problems, I don't think the solutions generated would have been as "good".
I'm also very uncomfortable with the question of how we can use this to augment our individual creativity. I think we need to be careful about who we mean when we say "our". Is each individual crowdworker having their creativity augmented? Certainly not. So, we've moved from an individually creative platform like Quirky, where as a whole the system may be inefficient, but each individual may benefit from going through the creative process, to an assembly line model, where the system is creative (and perhaps efficient), but each individual may or may not be performing a creative act, once decomposed into its constituent elements.
|
|
