Evaluating authoritative sources using social networks: an insight from Wikipedia

From WikiLit
Revision as of 20:26, January 30, 2014 by Fnielsen (Talk | contribs) (Text replace - "|collected_datatype=" to "|data_source=")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Publication (help)
Evaluating authoritative sources using social networks: an insight from Wikipedia
Authors: Nikolaos Th. Korfiatis, Marios Poulos, George Bokos [edit item]
Citation: Online Information Review 30 (3): 252-262. 2006.
Publication type: Journal article
Peer-reviewed: Yes
Database(s):
DOI: 10.1108/14684520610675780.
Google Scholar cites: Citations
Link(s): Paper link
Added by Wikilit team: Added on initial load
Search
Article: Google Scholar BASE PubMed
Other scholarly wikis: AcaWiki Brede Wiki WikiPapers
Web search: Bing Google Yahoo!Google PDF
Other:
Services
Format: BibTeX
Evaluating authoritative sources using social networks: an insight from Wikipedia is a publication by Nikolaos Th. Korfiatis, Marios Poulos, George Bokos.


[edit] Abstract

Purpose - The purpose of this paper is to present an approach to evaluating contributions in collaborative authoring environments, and in particular, Wikis using social network measures. Design/methodology/approach - A social network model for Wikipedia has been constructed, and metrics of importance such as centrality have been defined. Data has been gathered from articles belonging to the same topic using a web crawler, in order to evaluate the outcome of the social network measures in the articles. Findings - Finds that the question of the reliability regarding Wikipedia content is a challenging one and as Wikipedia grows, the problem becomes more demanding, especially for topics with controversial views such as politics or history. Practical implications - It is believed that the approach presented here could be used to improve the authoritativeness of content found in Wikipedia and similar sources. Originality/value - This work tries to develop a network approach to the evaluation of Wiki contributions, and approaches the problem of quality Wikipedia content from a social network point of view.

[edit] Research questions

"Wikipedia has internal mechanisms of managing those cases such as a permission ranking system, where a contributor is accredited by the level of participation in the shaping of the article, as well as a discussion tab on most of the articles or notifications and warnings regarding the content. Nevertheless, the research question looks at how to provide a clue to the credibility for an article based on the contributing authorities, and their acceptance by the community of their fellow contributors.

In this paper we present an initial attempt to model the problem towards providing an authoritative ranking mechanism based on social interaction data collected through the Wiki. Social interaction is approached from social communication facilitated by the Wikipedia platform (e.g. edits on edits) (Cobley, 1996). We then model the credibility of each contributor using the metric of centrality, thus producing an overall centrality measure for the article depicting the social activity/process that has taken place through the shaping of the article. We argue that this factor can be used as a metric of credibility, representing the article and the contributing authorities."

Research details

Topics: Computational estimation of trustworthiness [edit item]
Domains: Information systems, Sociology [edit item]
Theory type: Design and action [edit item]
Wikipedia coverage: Main topic [edit item]
Theories: "Graph Theory :

In classical social network models, the inner degree (the amount of edges coming into a node) represents the choices the actor has over a set of other actors. However, in our Wiki network model, the amount of incoming edges represents edits to the text; therefore, the metric of inner degree is the opposite, meaning that the person with the biggest inner degree has the biggest amount of objection/rejection in the contributor community, and thus receives a kind of negative evaluation from his/her fellow contributors. On the other hand, the outer-degree of the vertex represents edits/participation in several parts of the article, and thus gives a clue to the activity of the person in relation to the article and the domain. Mathematically, we can represent such formalism as follows: considering a graph representing the network of contributors for an article contributed in the Wiki, the contributor degree centrality – a contextualized expression of actor degree centrality – is a degree index of the adjacent connections between the contributor and others who edit the article. From graph theory, the outer degree of a vertex is the cumulative value of its adjacent connections: Equation 1 The adjacent xij represents the relational tie between the contributors and their contribution over the domain of the article. This also is characterized by the visibility of the contribution in the final article and can be either 1 or 0. To provide the centrality, we divide the degree with the highest obtained degree from the graph, which in graph theory is proved to be the number of remaining vertices (g) minus the self (g-1). Therefore, the contributor degree centrality can be calculated as: Equation 2" [edit item]

Research design: Mathematical modeling [edit item]
Data source: Wikipedia pages [edit item]
Collected data time dimension: Cross-sectional [edit item]
Unit of analysis: Article, Edit, User [edit item]
Wikipedia data extraction: Live Wikipedia [edit item]
Wikipedia page type: Article [edit item]
Wikipedia language: English [edit item]

[edit] Conclusion

"The question of the reliability regarding Wikipedia content is a challenging one. As long as the size of Wikipedia grows, the problem becomes more demanding, especially for topics with controversial views such as politics or history. Our study represents an early attempt at getting to the problem and thus working towards a more sophisticated solution to address it. However, there are a number of open issues that can extend the merit of this report.

The in-degree can be calculated using a more sophisticated factor, representing how much of the text contributed by one actor has been edited by another. In our case, we represent the editing or the objection by using a scale from 0 to 1, thus aggregating the factors using simple sums. A fuzzy operator could provide a solution for aggregating the results obtained by undertaking a fuzzy diff between the current version of the article and the version submitted. In that case, the social tie also needs to be expressed in terms of fuzziness, along with the relevant cases. Expressions of credibility using imprecise criteria (Sicilia and García, 2004, 2005) can also contribute to further advancement in that direction.

The organization of topics and the definition of inter-connections is also a matter for research, since there are related domains with contributing authorities. For instance, in the category of the social sciences, a contributor who edits the article of Adam Smith and has an acceptance factor can be retained on both the economics and philosophy domains, as an article about Adam Smith is represented in both. In that case, network modelling using two layer networks (document reference, authority reference) can enhance the trust of the contributions (Hess et al., 2006).

Furthermore, the measures developed and presented in this report do not actually measure the subjective quality of an article, since such a task is a cognitive process characterized by a high level of complexity. Those measures can contribute to providing an indicator of consensus related to an article, and thus assert that it does not provide controversial views or expressions of a small group of persons (especially in articles with political content). Thus, a level of neutrality expressed in the writing of the article is asserted.

Finally, specific attention should be given to the diffusion of different affiliations related to one actor. For example, a contributor may have many affiliations to unrelated subjects. This may imply that the contributor has knowledge of both fields, but in special topic cases (e.g. cardiology), the contribution in subjects such as Renaissance can be attributed as a non-expert one. Therefore, a classification of the competencies of each contributor may need to be promoted to strengthen their credibility and association with the subject or the article contributed."

[edit] Comments

"providing an authoritative ranking mechanism based on social interaction data, we measure the reliability of an article based on credibility of contributing authorities."


Further notes[edit]

Facts about "Evaluating authoritative sources using social networks: an insight from Wikipedia"RDF feed
AbstractPurpose - The purpose of this paper is to Purpose - The purpose of this paper is to present an approach to evaluating contributions in collaborative authoring environments, and in particular, Wikis using social network measures. Design/methodology/approach - A social network model for Wikipedia has been constructed, and metrics of importance such as centrality have been defined. Data has been gathered from articles belonging to the same topic using a web crawler, in order to evaluate the outcome of the social network measures in the articles. Findings - Finds that the question of the reliability regarding Wikipedia content is a challenging one and as Wikipedia grows, the problem becomes more demanding, especially for topics with controversial views such as politics or history. Practical implications - It is believed that the approach presented here could be used to improve the authoritativeness of content found in Wikipedia and similar sources. Originality/value - This work tries to develop a network approach to the evaluation of Wiki contributions, and approaches the problem of quality Wikipedia content from a social network point of view.ntent from a social network point of view.
Added by wikilit teamAdded on initial load +
Collected data time dimensionCross-sectional +
Commentsproviding an authoritative ranking mechanism based on social interaction data, we measure the reliability of an article based on credibility of contributing authorities.
ConclusionThe question of the reliability regarding The question of the reliability regarding Wikipedia content is a challenging one. As long as the size of Wikipedia grows, the problem becomes more demanding, especially for topics with controversial views such as politics or history. Our study represents an early attempt at getting to the problem and thus working towards a more sophisticated solution to address it. However, there are a number of open issues that can extend the merit of this report.

The in-degree can be calculated using a more sophisticated factor, representing how much of the text contributed by one actor has been edited by another. In our case, we represent the editing or the objection by using a scale from 0 to 1, thus aggregating the factors using simple sums. A fuzzy operator could provide a solution for aggregating the results obtained by undertaking a fuzzy diff between the current version of the article and the version submitted. In that case, the social tie also needs to be expressed in terms of fuzziness, along with the relevant cases. Expressions of credibility using imprecise criteria (Sicilia and García, 2004, 2005) can also contribute to further advancement in that direction.

The organization of topics and the definition of inter-connections is also a matter for research, since there are related domains with contributing authorities. For instance, in the category of the social sciences, a contributor who edits the article of Adam Smith and has an acceptance factor can be retained on both the economics and philosophy domains, as an article about Adam Smith is represented in both. In that case, network modelling using two layer networks (document reference, authority reference) can enhance the trust of the contributions (Hess et al., 2006).

Furthermore, the measures developed and presented in this report do not actually measure the subjective quality of an article, since such a task is a cognitive process characterized by a high level of complexity. Those measures can contribute to providing an indicator of consensus related to an article, and thus assert that it does not provide controversial views or expressions of a small group of persons (especially in articles with political content). Thus, a level of neutrality expressed in the writing of the article is asserted.

Finally, specific attention should be given to the diffusion of different affiliations related to one actor. For example, a contributor may have many affiliations to unrelated subjects. This may imply that the contributor has knowledge of both fields, but in special topic cases (e.g. cardiology), the contribution in subjects such as Renaissance can be attributed as a non-expert one. Therefore, a classification of the competencies of each contributor may need to be promoted to strengthen their credibility and association with the subject or the article contributed.
th the subject or the article contributed.
Data sourceWikipedia pages +
Doi10.1108/14684520610675780 +
Google scholar urlhttp://scholar.google.com/scholar?ie=UTF-8&q=%22Evaluating%2Bauthoritative%2Bsources%2Busing%2Bsocial%2Bnetworks%3A%2Ban%2Binsight%2Bfrom%2BWikipedia%22 +
Has authorNikolaos Th. Korfiatis +, Marios Poulos + and George Bokos +
Has domainInformation systems + and Sociology +
Has topicComputational estimation of trustworthiness +
Issue3 +
Pages252-262 +
Peer reviewedYes +
Publication typeJournal article +
Published inOnline Information Review +
Research designMathematical modeling +
Research questionsWikipedia has internal mechanisms of managWikipedia has internal mechanisms of managing those cases such as a permission ranking system, where a contributor is accredited by the level of participation in the shaping of the article, as well as a discussion tab on most of the articles or notifications and warnings regarding the content. Nevertheless, the research question looks at how to provide a clue to the credibility for an article based on the contributing authorities, and their acceptance by the community of their fellow contributors. In this paper we present an initial attempt to model the problem towards providing an authoritative ranking mechanism based on social interaction data collected through the Wiki. Social interaction is approached from social communication facilitated by the Wikipedia platform (e.g. edits on edits) (Cobley, 1996). We then model the credibility of each contributor using the metric of centrality, thus producing an overall centrality measure for the article depicting the social activity/process that has taken place through the shaping of the article. We argue that this factor can be used as a metric of credibility, representing the article and the contributing authorities. article and the contributing authorities.
Revid10,753 +
TheoriesGraph Theory :

In classical social neGraph Theory :

In classical social network models, the inner degree (the amount of edges coming into a node) represents the choices the actor has over a set of other actors. However, in our Wiki network model, the amount of incoming edges represents edits to the text; therefore, the metric of inner degree is the opposite, meaning that the person with the biggest inner degree has the biggest amount of objection/rejection in the contributor community, and thus receives a kind of negative evaluation from his/her fellow contributors. On the other hand, the outer-degree of the vertex represents edits/participation in several parts of the article, and thus gives a clue to the activity of the person in relation to the article and the domain. Mathematically, we can represent such formalism as follows: considering a graph representing the network of contributors for an article contributed in the Wiki, the contributor degree centrality – a contextualized expression of actor degree centrality – is a degree index of the adjacent connections between the contributor and others who edit the article. From graph theory, the outer degree of a vertex is the cumulative value of its adjacent connections: Equation 1 The adjacent xij represents the relational tie between the contributors and their contribution over the domain of the article. This also is characterized by the visibility of the contribution in the final article and can be either 1 or 0. To provide the centrality, we divide the degree with the highest obtained degree from the graph, which in graph theory is proved to be the number of remaining vertices (g) minus the self (g-1). Therefore, the contributor degree centrality can be calculated as: Equation 2entrality can be calculated as: Equation 2
Theory typeDesign and action +
TitleEvaluating authoritative sources using social networks: an insight from Wikipedia
Unit of analysisArticle +, Edit + and User +
Urlhttp://dx.doi.org/10.1108/14684520610675780 +
Volume30 +
Wikipedia coverageMain topic +
Wikipedia data extractionLive Wikipedia +
Wikipedia languageEnglish +
Wikipedia page typeArticle +
Year2006 +