|Quality assessment process in Wikipedia's Vetrina: the role of the community's policies and rules|
|Citation:||Observatorio (OBS*) 3 (1): . 2009 March.|
|Publication type:||Journal article|
|Google Scholar cites:||Citations|
|Added by Wikilit team:||Added on initial load|
|Article:||Google Scholar BASE PubMed|
|Other scholarly wikis:||AcaWiki Brede Wiki WikiPapers|
|Web search:||Bing Google Yahoo! — Google PDF|
The increasing growth of Wikipedia poses many questions about its organizational model and its development as a free-open knowledge repository. Yochai Benkler describes Wikipedia as a CBPP (commons-based peer production) system: a platform which enables users to easily generate knowledge contents and to manage them collaboratively and on free-voluntary basis. Quality is one of the main concerns related to such a system. How would a CBPP environment guarantee at the same time the openness of its organization and a good level of accreditation? The paper offers an overview of the quality assessment processes in it.wiki's Vetrina section. It also suggests an explanation to quality assessment which questions Benkler's hypothesis. Thanks to a qualitative analysis carried out through in-depth interviews to Wikipedia users and through a period of ethnographic observation, the paper outlines Vetrina's organization and the factors related to the evaluation of quality contents.
"How would a CBPP environment guarantee at the same time the openness of its organization and a good level of accreditation? The paper offers an overview of the quality assessment processes in it.wiki’s Vetrina section. It also suggests an explanation to quality assessment which questions Benkler’s hypothesis. Thanks to a qualitative analysis carried out through in-depth interviews to Wikipedia users and through a period of ethnographic observation, the paper outlines Vetrina’s organization and the factors related to the evaluation of quality contents.
My paper wants to verify Benkler’s hypothesis through a qualitative analysis focused on one of the most popular quality assessment processes in Wikipedia: the Featured Articles section (Vetrina in Italian, http://it.wikipedia.org/wiki/Wikipedia:Vetrina). My survey doesn’t concentrate on the content’s quality itself but it reflects on the organizational and social processes involved in Vetrina’s quality assessment: it aims to verify whether quality is the output of the articulations of recognized roles and functions as assessed by Benkler, or it is more related to social policies and rules shared by Wikipedia community."
|Topics:||Featured articles, Social order|
|Wikipedia coverage:||Main topic|
|Theories:||"Others (Anthony et. al., 2005) relate quality to the openness of the system: the spontaneous and self selected recruitment which distinguish the participation in Wikipedia, the policy of free access and the user-friendliness of the technological system encourage contributions from users with heterogeneous competences and cultural backgrounds. According to Anthony’s analysis Wikipedia users could be of two types: Zealots, the registered users who constantly submit contents and systematically help to revise and update articles, and Good Samaritans, people who contribute sporadically, most of the time in anonymous way. Nevertheless Good Samaritans prove to be good contributors in the project. Through the evaluation of the quantity of contents retained from a user contribution, the authors formulate a quantitative measure of quality contribution of the two users’ typologies and they conclude that Good Samaritans are even better contributors than Zealots. Contrary to the arguments which relate good quality in Open Source systems to reputation issues (Ghosh, Prakash 2000; Lerner, Tirole 2002; Lakhani, Von Hippel 2002), - a registered user with a constant presence in the project is more likely to submit the best content in order to reinforce his reputation –, Anthony et al. find that quality in Wikipedia is more linked to the openness of its organization and technological system. Such an openness encourages in fact, both the Zealots to contribute and improve their competences through a constant participation, and the one-time anonymous users to enrich in unforeseeable way, the development of a common knowledge repository.
|Research design:||Content analysis, Ethnography|
|Data source:||Direct observation, Interview responses|
|Collected data time dimension:||Longitudinal|
|Unit of analysis:||User, Website|
|Wikipedia data extraction:||Live Wikipedia|
|Wikipedia page type:||Article, Discussion and Q&A, Quality management|
"The paper tries to compare Wikipedia’s Vetrina process to that of CBPP systems since they both involve web based decentralized activities, voluntary collaboration, they aim at creating a common repository of knowledge and both of them do without an authoritative editorial board of experts for contents elaboration.
Notwithstanding those shared features, Vetrina’s organization and policies differ widely from the experience of a CBPP as for example Slashdot. In Slashdot, quality is the output of a complex subdivision of recognized roles and functions: in some cases roles are even institutionalized with a formal payment. On the contrary, the survey reveals that Vetrina’s quality assessment process develops on a peer and self-selected basis. Every single user can become an “editor” or an “author” and in a reasonable period of time he/she could become more and more ommitted in writing and reviewing articles for Vetrina’s selection process (Bryant et al., 2005). Even at the beginning of this involvement in Wikipedia’s activities, users competences and skills are self-assessed and not evaluated by automated reputation systems as in Slashdot. At the same time, Wikipedians tend to reject the recognition of roles in the matter of content quality. Stating the role of “an individual author” or “Vetrina director” might encourage in fact, the emerging of unilateral points of view in spite of one of the project’s pillars: the NPOV (Neutral Point of View). It.wiki community seems to avoid any kind of individual authorship and tend to hold in higher regard technical roles as Administrators, Burocrati and Checkusers who are devoted to daily management activities. Those roles, especially in the words of the directly involved people, are considered the most important for the encyclopaedia’s development and maintenance.
I can conclude that, contrary to Benkler’s hypothesis, quality in Vetrina doesn’t depend on a progressive definition of roles and competences as observed in other CBPP experience. Nevertheless Wikipedia is very far from being a semi-anarchic system: the project shows a structured and dynamic social system which seems to evolve toward an even more complex organization based upon multiple policies and rules (Kittur et al., 2007b). Vetrina represents a sub-world which reflects in its articulation the widest system it belongs to. In the development of this sub-system, policies and rules defined by the community have a strategic role. Quality indeed seems to be, as also Viégas (Viégas et al., 2004, 2007), Lih, (Lih, 2004), Emigh and Herring (Emigh, Herring, 2005) observed, the result of those principles observation and common acceptance, the output of a community culture considered as the whole of rules, values and procedures rather than the product of a formal organization of roles and functions. Wikipedia’s policies represent in fact the fundamental “pillars” of the project while an egalitarian ethic leads the community actions against any kind of individual “authorship” or unilateral control on information. Procedures are also very important: through the subsequent phases of Exam and Report articles can reach a recognized quality standard while the peer review process and the voting aimed at the contents improvement are the necessary phases for the articles assessment. A set of rules finally, as for example, those related to the final voting, guarantee the reliability and precision of Vetrina procedures.
Moving toward higher quality standards, it.wiki reveals a high degree of flexibility: as the survey shows up, the number of procedures has significantly increased as well as the definition of new rules and policies. Nevertheless this bureaucratization process was led by the open dialogue and coordination among Wikipedians and not by an authoritarian initiative: this process grew up in a “rational” way according to the community principles and values. Procedures and rules have been defined according to the openness and collaboration policies which represent the distinct features of Wikipedia project. The survey results give a better understanding of Wikipedia social organization and may put a new light on the transformation of CBPP systems on line: it.wiki experience shows that open knowledge on line communities could reach higher quality standards even without recognized roles and institutionalised rewards. Emergent coordination and self-selected competences regulated by shared social rules and policies may in fact play a fundamental role in the development and management of common knowledge creation."