k
New Member
Posts: 9 
|
Post by k on Apr 24, 2016 0:40:34 GMT
Summary:This paper reviews a wide range of research on Crowd Work (defined as the performance of online tasks by crowd workers who are financially compensated by requesters) and explores issues necessary to ensure that crowd work is valuable to both requesters and workers. Considering crowd work from the perspectives of both the worker and requester, the authors develop an analytic framework concentrated on the question, “Can we foresee a future crowd workplace in which we would want our children to participate?”. The authors argue that crowd work can be valuable to both the worker and requester, but that design choices require careful consideration to avoid a trajectory that is exploitative and/or leads to low-quality work. The paper draws from theories in organizational behavior and distributing computing as well as a survey of crowd workers to frame 12 research foci towards which it is argued that crowd work research should be directed to steer crowd work towards positive results. Future of Crowd Work Processes- Workflow - How can tasks be more effectively broken down and distributed across workers?
- Task Assignment - How can particular tasks be efficiently distributed to appropriate workers?
- Hierarchy - Can hierarchical business practices be implemented in crowd work?
- Realtime Crowd Work- Can real-time crowd work work at scale and be used to meet deadlines?
- Synchronous Collaboration - How can cooperation between workers be facilitated?
- Quality Control - How do we effectively recognize and reward quality at scale?
The Future of Crowd Computation- Crowds Guiding AIs - Can crowd actions responses be used to train AIs to complete similar tasks?
- AIs Guiding Crowds - Can AI be used to better manage crowd workers?
- Crowdsourcing Platforms - Are there alternative to existing crowd work platforms that could improve crowd work?
- The Future of Crowd Workers
- Job Design - How can crowd work tasks be designed to to improve worker satisfaction?
- Reputation and Credentials - How can reputations be established for both workers and requesters?
- Motivation and Rewards - How can motivations beyond financial incentives be built into crowd work?
The authors present three design goals that would incorporate the research foci with specific goals. - Create Career Ladders - Create a system that allows for evolving jobs that crowd worker can move up through.
- Improve Task Design through Better Communication - Incorporate more feedback and interaction between requesters and workers in crowd work platforms.
- Facilitate Learning - Engage workers with tasks that push them to learn new skills over time.
Questions:- Are you convinced by the authors that crowd work can be positive for both requesters and workers?
- Do you think the areas of research the authors identify--processes, computation, and workers--can be adequately addressed to deliver a positive answer to the authors’ question about the desirability of crowd work for future generations? Why or why not?
- Reflecting on the class discussion about Vannevar Bush’s article ‘As We May Think!’, how does this article make a research contribution and what criteria should it be evaluated against?
|
|
|
|
Post by sciutoalex on Apr 24, 2016 20:17:05 GMT
Are you convinced by the authors that crowd work can be positive for both requesters and workers?
Last week Uber settled with California and Massachusetts drivers agreeing to pay $100 million dollars to 350,000 workers. The drivers' status will remain as independent contractors—ahem crowdworkers. Each worker will get approximately $250. Uber will create "drivers' associations" and appeals processes, but these will not include collective bargaining. This has been hailed as a significant victory for Uber as it doesn't threaten Uber's core business strategy of depriving people who work for them of paid time off and retirement accounts (I'm biased).
Reading this paper with the backdrop of this news, I'm struck at how the details crowdworkers' futures are not in the hands of HCI researchers. Suggesting that gamification of career or adding in learning opportunities will make crowdsourcing a career I'd be happy that my future children have is laughable. None of us reading this thread would be happy with these careers and we're hypocrites for asking anyone else to have this insecure, badly paying life. This paper was written in 2013, three years ago. I don't blame the researchers for not seeing how quickly economics would influence the adoption of crowdsourcing.
So what should HCI be doing? First, realize and reckon with the fact that crowdsourcing is another step in a long process of using technology to weaken the security of workers for the sake of greater profits/efficiency. After that, I think HCI should consider how to design for collective action, how to design in the face of economic disparity, how to design in a context where companies that harness crowdsourced work are at a large advantage above their workers. Crowdsourcing is becoming a wide-spread method of production. Understanding how we might counter it, not just make it better, is something HCI should be thinking about.
We all live in Pittsburgh. At Carnegie Steel's height, Pittsburgh was its company town. When workers tried to unionize in 1892, the Homestead Steel Strike commenced which ended with 12 killed and over 20 wounded. The unionizers lost. The tension between "requesters" and "workers" is an old one, and HCI's decision to think only in terms of interesting research and ignore historical struggles makes it more likely to contribute to a future that no one wants to live in.
|
|
toby
New Member
Posts: 21
|
Post by toby on Apr 25, 2016 14:32:23 GMT
Are you convinced by the authors that crowd work can be positive for both requesters and workers?Last week Uber settled with California and Massachusetts drivers agreeing to pay $100 million dollars to 350,000 workers. The drivers' status will remain as independent contractors—ahem crowdworkers. Each worker will get approximately $250. Uber will create "drivers' associations" and appeals processes, but these will not include collective bargaining. This has been hailed as a significant victory for Uber as it doesn't threaten Uber's core business strategy of depriving people who work for them of paid time off and retirement accounts (I'm biased). Reading this paper with the backdrop of this news, I'm struck at how the details crowdworkers' futures are not in the hands of HCI researchers. Suggesting that gamification of career or adding in learning opportunities will make crowdsourcing a career I'd be happy that my future children have is laughable. None of us reading this thread would be happy with these careers and we're hypocrites for asking anyone else to have this insecure, badly paying life. This paper was written in 2013, three years ago. I don't blame the researchers for not seeing how quickly economics would influence the adoption of crowdsourcing. So what should HCI be doing? First, realize and reckon with the fact that crowdsourcing is another step in a long process of using technology to weaken the security of workers for the sake of greater profits/efficiency. After that, I think HCI should consider how to design for collective action, how to design in the face of economic disparity, how to design in a context where companies that harness crowdsourced work are at a large advantage above their workers. Crowdsourcing is becoming a wide-spread method of production. Understanding how we might counter it, not just make it better, is something HCI should be thinking about. We all live in Pittsburgh. At Carnegie Steel's height, Pittsburgh was its company town. When workers tried to unionize in 1892, the Homestead Steel Strike commenced which ended with 12 killed and over 20 wounded. The unionizers lost. The tension between "requesters" and "workers" is an old one, and HCI's decision to think only in terms of interesting research and ignore historical struggles makes it more likely to contribute to a future that no one wants to live in. ----- @alex made a great point by saying turker is an undesirable career as it's insecure and badly paying. However, I would argue that it's mostly about badly paying (for a U.S. standard) and less so for insecurity. Many people are happy about higher-paying jobs with low job security - independent consultant, sales agent etc, but I doubt if many people (in the U.S.) would be happy being a full-time turker if we ensure job security and pay for insurances and benefits, but pay $1-2 per hour. I went on to find out who are the turkers. I always thought most turkers are foreign workers from developing countries - where $2 per hour is actually not a bad wage. Surprisingly, when I browse some turker forum, many of them are U.S. based. It turns out that many of them just use MTurk in spare time as a supplement to their main income, where you can just make use of the 1 or 2 hours in commuting, after dinner, or slack off at work to make an extra $200 for the month. For this case I think it's perfectly reasonable to be a casual turker then, as the turker can make money for the time that would be "wasted" otherwise anyway. I would argue that non-expert turker (at least for U.S. based ones) will not, and should not become a viable full-time job option, since the causal turkers consider their time to have a much smaller opportunity cost than full-time turkers and thus are willing to take the job for lower price. Since the majority of turker tasks are not time-dependent, it would be a misallocation of resource to use big chunk of day time to do those tasks. And this is very different from Uber -- as we'll need people to drive Uber from 9-5 in the day. |
|
|
|
Post by JoselynMcD on Apr 25, 2016 15:53:23 GMT
**Are you convinced by the authors that crowd work can be positive for both requesters and workers?**
This is one of those 'context is everything' kind of research areas. I think yes, hypothetically there could be a system that allows for a positive experience for both requesters and works, but what kind of world would have to exist for this kind of work to be coveted or sought after by the general public - it's harrowing for me to even picture it. Colleagues in this department and I were just talking about the inevitability of a global decrease of people working in job sectors such as transportation, rote and repetitive factory work, clerical services, and potentially even retail in the coming decade because of the increase in capability of electronic automized systems. To this end, a world in which a significant portion of the population doesn't report to work everyday and thus might turn to mechanical turk work for income isn't too hard to fathom.
In picturing (as Alex would put it) this kind of dystopia, I'm grateful that research around attempting to making this work more bearable is being done. However, I find that often optimal gets confused with good is a fallacy, especially when it comes to human beings. I won't attempt to break down here all the reasons why I think that most people, if asked, would reject participation on the current* MTurk as a reasonable expectation of their time. I highlight current because as Nathan pointed out in a recent discussion, the current structure, pay, aesthetic, design, of Amazon's Mechanical Turk isn't necessarily the only way this sort of system need be. With that in mind, coupled with the bleak outcome for many people in sectors to be outsourced to electronic systems, I believe that a whole bevy of design solutions, like the ability to receive kudos, learn new skills, mentor, be mentored, etc. (in addition to a guaranteed monthly income, if I get to have my way) need to be in place to ensure that we don't just have an optimal, but a human-oriented MTurk system.
|
|
nhahn
New Member
Posts: 22
|
Post by nhahn on Apr 25, 2016 18:03:12 GMT
Following on this discussion, do you think any type of task can be "microtasked" and distributed? I feel that a lot of research is moving towards this direction, where microtasks are consider the optimal situation, due to scalability benefits. However, are these microtask workflows really the optimal solution? Or are they just good from some tasks but worse for others? In their current version, I don't think they leverage expertise well. With general market platforms (like mTurk) the level of expertise is limited. So tasks have to be designed for anyone to perform, and usually for an worker to get started, they have to be introduced to a lot of new, unfamiliar information in order to effectively contribute. Experts are beneficial, not only for their wealth of knowledge, but a lot of times also for their speed. One view of expertise is a transition of information from the cognitive system to the perceptual system. For example, expert chess players can recognize setups extremely quickly and know the appropriate response, often without having to think about the situation significantly. If we have these set workflows for microtasking that doesn't take advantage of this situation recognition ability of experts, that seems like a huge loss in terms of wasted work. I could see one potential solution -- training AI task managers to recognize common problems, and produce more streamlined workflows based on previous performance.
|
|
|
|
Post by judithodili on Apr 25, 2016 22:42:45 GMT
I'm not sure - this paper reminded me a lot of my experience using turk. I don't know how people do it - for cents on the dollar, i had someone complete my survey (with long free text feedback) 66 times!! He made a total of $12 on my task. It makes no sense that people do this outside of wanting to churn out a lot of low quality work with the time to make something that can buy them lunch. Some people genuinely share very good feedback so I'm not sure if they feel some kind of civic duty to do so or they just legit do this for a living. The conversation about balancing the quality of work vs rewards is always going to be interesting... The point is.. I dont see how this can be beneficial to the workers at all... Except for low 10 second tasks, I suspect that there'll be a lot of noise in the data collected by requesters.
|
|
|
|
Post by jseering on Apr 26, 2016 0:39:58 GMT
I guess the economist's argument would be that people wouldn't be participating in these services if they didn't feel like they were better off as a result, but I'm a little uncomfortable with researchers taking advantage of this for the sake of experimenting cheaply. I guess low cost per paper is desirable, but it feels a little... colonialist? It's probably better at least in the sense that not all of our participants are educated coastal US 20-year-olds. I think that crowd work can be positive for all, but I'm not exactly sure how.
|
|
mkery
New Member
Posts: 22 
|
Post by mkery on Apr 26, 2016 1:25:56 GMT
I cannot help but be firmly in the camp “crowdsourcing is bad for society”. Why are there not regulations of pay? Minimum wage exists for a reason, and should always be upheld. For this reason, among many, I am uncomfortable with many research papers including those in HCI that rely on crowdsourcing and emphasize cost effectiveness. If a paper promises “$8 to fix a software bug” through decomposing this task into crowd work, this is highly useful, but what humans beings am I exploiting by using such a system? If an accessibility tool creates an affordable system to read information to the blind, is HCI hurting a crowd to help an individual? While these remarks may seem on the paranoid end of things, I feel HCI has the need to be paranoid about these considerations as a field driving crowdsourcing research.
Also, is not one side-goal of reducing everything into unskilled labor, the ability to train AI to handle the decomposed tasks? While our society may be moving towards AI replacing many jobs, I strongly feel crowdsourcing should be watched carefully for human dignity in the meantime.
|
|
judy
New Member
Posts: 22
|
Post by judy on Apr 26, 2016 2:07:36 GMT
I was going to write something glib like, " sciutoalex: mic drop." But then I read mkery and she took it a step further. Nice job! I am, however, glad that this paper exists and that researcher are trying to think about the future of work in human terms. But if there is no mechanism to organize workers and speak to their interests (Unions are one way we do this, but there are other less formal models) than crowdwork is not a viable solution. In an effort to contribute something more to this conversation, I want to point to the kinds of work that tons of people on the internet find implicit value in: creative work--creating memes, joining fandoms, microblogging, open source technologies. All of these forms have value for the worker because they are based on the worker's interests, talents and identity. Often it's intensely personal, and even if it's not, the worker receives public and almost instant recognition for their work. Why are we willing to do this work for free? What is the value of this work? What can we learn about this type of creative work that may inform a more economically viable model of online work? |
|
|
|
Post by mmadaio on Apr 26, 2016 2:44:28 GMT
The authors seem to acknowledge the danger of crowdwork systems that lead to "mechanized workers" or "people act[ing] as computational components", and give cursory acknowledgment to the importance of social mobility, job security, and building marketable skills, but the focus of the paper, even within the section on "The future of Crowd Workers", is on how to best tailor the Job Design, Reputation, and Rewards of a crowd-work platform to engineer a more efficient and reliable crowd-work experience, not to qualitatively improve the work experience for the worker on their terms. I am HIGHly skeptical that the authors of this paper would truly be "proud" if their children decided to forego a more traditional career for the piecemeal instability of crowd-work, at least for anything beyond a summer or winter job supplement during school. Sure. There's the argument that Joseph mentioned, that, much like Uber, people aren't compelled to take these jobs. And yet. How is this affecting the larger cultural understanding of what work looks like, and what an employer's responsibility is to their employees? Lily Irani has a fabulous paper on this topic: nms.sagepub.com/content/early/2013/11/19/1461444813511926.abstract |
|
|
|
Post by julian on Apr 26, 2016 2:45:24 GMT
Are you convinced by the authors that crowd work can be positive for both requesters and workers?So what should HCI be doing? First, realize and reckon with the fact that crowdsourcing is another step in a long process of using technology to weaken the security of workers for the sake of greater profits/efficiency. After that, I think HCI should consider how to design for collective action, how to design in the face of economic disparity, how to design in a context where companies that harness crowdsourced work are at a large advantage above their workers. Crowdsourcing is becoming a wide-spread method of production. Understanding how we might counter it, not just make it better, is something HCI should be thinking about. I agree completely with Alex. Unfortunately crowdsourcing is pretty much equivalent of cheap unskilled(not-necessarily) labor. In this case is even worst because crowdworkers are nameless, faceless and likely have no easy way to connect to other workers, no way to complain or sue. This is a rather perfect system for a company to exploit workers and not paying any benefits. Now, if crowdsourcing was a more fair system, then what the authors propose makes more sense, however, this would mean a higher pay, benefits, etc which basically may take away most of the economic incentive for using crowdsourcing in the first place. I'm wondering whether crowdsourcing could exist at all under fair conditions for the workers. |
|
|
|
Post by xiangchen on Apr 26, 2016 2:52:01 GMT
I believe there is a type of crowd work that can benefit both the requesters and the workers. Such a type of crowd work will probably resemble wikis, or the Polymath project described in the other paper. It will be domain-specific. It will require 'workers' to communicate with and learn from each other before or besides making their own contribution.
That said, and this paper read, I am not sure this will be necessarily be _the_ future of crowd work. It will be probably be part of the future. Or perhaps a pretty big part. But it seems improbably that the traditional model (e.g., crowd workers doing isolated low level micro tasks) will die out. I believe between AI trying to encroach humans' job market and crowd workers collectively triumphing professional individuals, the simplest model will still find its place.
|
|
|
|
Post by francesx on Apr 26, 2016 3:27:09 GMT
RE: Are you convinced by the authors that crowd work can be positive for both requesters and workers?
I agree with Alex and with what other people say here. This sounds like a very scary future. How is this crowd work future different from those companies in 3rd world countries who get very cheap labor to produce clothes and shoes and electronics?
Rather than a permanent job, I would support crowd work as a temporary job or part-time job, but definitely not a full time one.
|
|
|
|
Post by cgleason on Apr 26, 2016 4:18:38 GMT
Re: Are you convinced by the authors that crowd work can be positive for both requesters and workers?
Yes. The problem is that we have been treating crowd workers as a pool of non-specialized labor to transition from a human knowledge worker society to one automated through AI. As through all history, we will end up moving to more an more specialized labor, so I agree that learning and career paths in crowd work is necessary. I don't like to think of it as gamification, because that term seems to convey a sense of "empty value". If this career path is not providing crowd workers with autonomy and actual fulfillment, then it's done incorrectly.
As someone who has done work as a "crowd-worker" software programmer (remote independent contractor), this can be incredibly rewarding. Building something for a client (requestor) feels very creative, as you get to design the final product in your own way. The difference between this and current crowd work, is that my "requestor" sees me as a skilled worker and doesn't reduce my job to a "microtask". That is where the future of crowd work is, I believe. Highly skilled, extremely specialized, remote consultants.
|
|
vish
New Member
Posts: 17
|
Post by vish on Apr 26, 2016 4:41:22 GMT
|
|
?
