Harnessing the wisdom of crowds in Wikipedia: quality through coordination

From WikiLit
Jump to: navigation, search
Publication (help)
Harnessing the wisdom of crowds in Wikipedia: quality through coordination
Authors: Aniket Kittur, Robert E. Kraut [edit item]
Citation: CSCW '08 Proceedings of the 2008 ACM conference on Computer supported cooperative work  : 37-46. 2008 November 8-12. San Diego, CA, United states. Association for Computing Machinery.
Publication type: Conference paper
Peer-reviewed: Yes
Database(s):
DOI: 10.1145/1460563.1460572.
Google Scholar cites: Citations
Link(s): Paper link
Added by Wikilit team: Added on initial load
Search
Article: Google Scholar BASE PubMed
Other scholarly wikis: AcaWiki Brede Wiki WikiPapers
Web search: Bing Google Yahoo!Google PDF
Other:
Services
Format: BibTeX
Harnessing the wisdom of crowds in Wikipedia: quality through coordination is a publication by Aniket Kittur, Robert E. Kraut.


[edit] Abstract

Wikipedia's success is often attributed to the large numbers of contributors who improve the accuracy, completeness and clarity of articles while reducing bias. However, because of the coordination needed to write an article collaboratively, adding contributors is costly. We examined how the number of editors in Wikipedia and the coordination methods they use affect article quality. We distinguish between explicit coordination, in which editors plan the article through communication, and implicit coordination, in which a subset of editors structure the work by doing the majority of it. Adding more editors to an article improved article quality only when they used appropriate coordination techniques and was harmful when they did not. Implicit coordination through concentrating the work was more helpful when many editors contributed, but explicit coordination through communication was not. Both types of coordination improved quality more when an article was in a formative stage. These results demonstrate the critical importance of coordination in effectively harnessing the wisdom of the crowd" in online production environments.

[edit] Research questions

"The present research uses longitudinal data to examine the conditions under which adding contributors to a Wikipedia article improves its quality. It shows that the effectiveness of adding contributors is critically dependent on the degree and type of coordination those contributors use, as well as the life cycle of the article and the interdependence of the tasks involved in editing it."

Research details

Topics: Antecedents of quality, Quality improvement processes [edit item]
Domains: Information systems [edit item]
Theory type: Analysis [edit item]
Wikipedia coverage: Main topic [edit item]
Theories: "Theories of group coordination suggest a basic distinction

between explicit coordination, based on direct communication and verbal planning, and implicit coordination, based on workgroup structure, unspoken expectations and shared mental models of the task to be accomplished [28][40]. Below we examine evidence of both types of coordination in Wikipedia, focusing on direct communication and workgroup structure." [edit item]

Research design: Econometrics and time series [edit item]
Data source: Wikipedia pages [edit item]
Collected data time dimension: Longitudinal [edit item]
Unit of analysis: Article [edit item]
Wikipedia data extraction: Dump [edit item]
Wikipedia page type: Article [edit item]
Wikipedia language: Not specified [edit item]

[edit] Conclusion

"However, adding more editors to a page seems to improve its quality only when appropriate coordination techniques are used. Having more editors work on an article was associated with increases in article quality only if the editors used implicit coordination, so that most of the work was done by a small subset of them, with others playing a supporting role. Having more editors was not associated with improvements in article quality when the work was distributed evenly among editors or when they used explicit communication on the article talk page to coordinate. Phrased another way, both implicit coordination, through editor concentration, and explicit coordination, through communication, were valuable and were generally associated with improvement in article quality during the periods when they were used. However, implicit coordination was especially valuable for articles and time periods with many contributors, while explicit coordination was especially valuable for articles and time periods with few contributors. In addition, this research has demonstrated that both implicit and explicit coordination have stronger associations with increases in article quality early in an article’s life cycle, when the article is young and has little content. This is the period when tasks are most interdependent and coordination needs are highest, and this is when the article’s creator(s) need to provide a structure for the article to which others can contribute. For example, our data suggest that it is important to have a small number of contributors setting the direction, structure, and scope of the article at the beginning of its life cycle, either implicitly by actually doing the writing or by explicitly communicating and coordinating. As the article matures and coordination requirements ease, tasks may be more effectively distributed to a larger group of contributors. It is no surprise that articles in which the authors communicate with each other on the article’s talk page improve more than articles in which editors work independently, each making a contribution without discussing changes or getting advice from collaborators. Interpersonal communication is perhaps the most general coordination technique available and is the paradigmatic case of what March and Simon (1958) describe as "coordination through mutual adjustment," in which members reveal their current states and intentions and adjust their behavior to others’ goals and actions [33]. Decades of research in organizations show that communication as the basis for coordination is especially important in tasks that are highly uncertain, unconstrained, and subject to many changes (e.g., [18]). These conclusions are consistent with the observations in the current research that explicit communication through coordination is most beneficial in an article’s formative stages, when its structure is highly unconstrained. In the beginning, no one knows either the content or the structure of an article, but later in its life cycle the existing material constrains what new editors can write. The route by which the concentration of work leads to better coordination and improvements in quality is less clear. One possibility is that concentrating editing in fewer editors may enable a small core of committed contributors to focus on complex, highly interdependent tasks while allowing infrequent contributors to add value on simple, stand-alone tasks or those that benefit from diverse knowledge. For example, it may be effective for a small core group of users to work on tasks such as organizing an article and making it cohesive, whereas tasks with low coordination requirements—such as fixing grammar, combating vandalism, or creating links—may be more effectively distributed to peripheral contributors. This strategy may be an efficient way to take advantage of a large number of contributors while at the same time minimizing process losses."

[edit] Comments


Further notes[edit]

Facts about "Harnessing the wisdom of crowds in Wikipedia: quality through coordination"RDF feed
AbstractWikipedia's success is often attributed toWikipedia's success is often attributed to the large numbers of contributors who improve the accuracy, completeness and clarity of articles while reducing bias. However, because of the coordination needed to write an article collaboratively, adding contributors is costly. We examined how the number of editors in Wikipedia and the coordination methods they use affect article quality. We distinguish between explicit coordination, in which editors plan the article through communication, and implicit coordination, in which a subset of editors structure the work by doing the majority of it. Adding more editors to an article improved article quality only when they used appropriate coordination techniques and was harmful when they did not. Implicit coordination through concentrating the work was more helpful when many editors contributed, but explicit coordination through communication was not. Both types of coordination improved quality more when an article was in a formative stage. These results demonstrate the critical importance of coordination in effectively harnessing the wisdom of the crowd" in online production environments. crowd" in online production environments.
Added by wikilit teamAdded on initial load +
Collected data time dimensionLongitudinal +
ConclusionHowever, adding more editors to a page seeHowever, adding more editors to a page seems to improve its quality only when appropriate coordination techniques are used. Having more editors work on an article was associated with increases in article quality only if the editors used implicit coordination, so that most of the work was done by a small subset of them, with others playing a supporting role. Having more editors was not associated with improvements in article quality when the work was distributed evenly among editors or when they used explicit communication on the article talk page to coordinate. Phrased another way, both implicit coordination, through editor concentration, and explicit coordination, through communication, were valuable and were generally associated with improvement in article quality during the periods when they were used. However, implicit coordination was especially valuable for articles and time periods with many contributors, while explicit coordination was especially valuable for articles and time periods with few contributors. In addition, this research has demonstrated that both implicit and explicit coordination have stronger associations with increases in article quality early in an article’s life cycle, when the article is young and has little content. This is the period when tasks are most interdependent and coordination needs are highest, and this is when the article’s creator(s) need to provide a structure for the article to which others can contribute. For example, our data suggest that it is important to have a small number of contributors setting the direction, structure, and scope of the article at the beginning of its life cycle, either implicitly by actually doing the writing or by explicitly communicating and coordinating. As the article matures and coordination requirements ease, tasks may be more effectively distributed to a larger group of contributors. It is no surprise that articles in which the authors communicate with each other on the article’s talk page improve more than articles in which editors work independently, each making a contribution without discussing changes or getting advice from collaborators. Interpersonal communication is perhaps the most general coordination technique available and is the paradigmatic case of what March and Simon (1958) describe as "coordination through mutual adjustment," in which members reveal their current states and intentions and adjust their behavior to others’ goals and actions [33]. Decades of research in organizations show that communication as the basis for coordination is especially important in tasks that are highly uncertain, unconstrained, and subject to many changes (e.g., [18]). These conclusions are consistent with the observations in the current research that explicit communication through coordination is most beneficial in an article’s formative stages, when its structure is highly unconstrained. In the beginning, no one knows either the content or the structure of an article, but later in its life cycle the existing material constrains what new editors can write. The route by which the concentration of work leads to better coordination and improvements in quality is less clear. One possibility is that concentrating editing in fewer editors may enable a small core of committed contributors to focus on complex, highly interdependent tasks while allowing infrequent contributors to add value on simple, stand-alone tasks or those that benefit from diverse knowledge. For example, it may be effective for a small core group of users to work on tasks such as organizing an article and making it cohesive, whereas tasks with low coordination requirements—such as fixing grammar, combating vandalism, or creating links—may be more effectively distributed to peripheral contributors. This strategy may be an efficient way to take advantage of a large number of contributors while at the same time minimizing process losses.t the same time minimizing process losses.
Conference locationSan Diego, CA, United states +
Data sourceWikipedia pages +
Dates8-12 +
Doi10.1145/1460563.1460572 +
Google scholar urlhttp://scholar.google.com/scholar?ie=UTF-8&q=%22Harnessing%2Bthe%2Bwisdom%2Bof%2Bcrowds%2Bin%2BWikipedia%3A%2Bquality%2Bthrough%2Bcoordination%22 +
Has authorAniket Kittur + and Robert E. Kraut +
Has domainInformation systems +
Has topicAntecedents of quality + and Quality improvement processes +
MonthNovember +
Pages37-46 +
Peer reviewedYes +
Publication typeConference paper +
Published inCSCW '08 Proceedings of the 2008 ACM conference on Computer supported cooperative work +
PublisherAssociation for Computing Machinery +
Research designEconometrics and time series +
Research questionsThe present research uses longitudinal datThe present research uses longitudinal data to examine the

conditions under which adding contributors to a Wikipedia article improves its quality. It shows that the effectiveness of adding contributors is critically dependent on the degree and type of coordination those contributors use, as well as the life cycle of the article and the interdependence of the tasks involved in editing it.dence of the tasks

involved in editing it.
Revid10,800 +
TheoriesTheories of group coordination suggest a bTheories of group coordination suggest a basic distinction

between explicit coordination, based on direct communication and verbal planning, and implicit coordination, based on workgroup structure, unspoken expectations and shared mental models of the task to be accomplished [28][40]. Below we examine evidence of both types of coordination in Wikipedia, focusing on direct communication and workgroup structure.ect

communication and workgroup structure.
Theory typeAnalysis +
TitleHarnessing the wisdom of crowds in Wikipedia: quality through coordination
Unit of analysisArticle +
Urlhttp://dl.acm.org/citation.cfm?id=1460572 +
Wikipedia coverageMain topic +
Wikipedia data extractionDump +
Wikipedia languageNot specified +
Wikipedia page typeArticle +
Year2008 +