Requirements-oriented methodology for evaluating ontologies

From WikiLit
Jump to: navigation, search
Publication (help)
Requirements-oriented methodology for evaluating ontologies
Authors: Jonathan Yu, James A. Thom, Audrey Tam [edit item]
Citation: Information Systems 34 (8): 686-711. 2009.
Publication type: Journal article
Peer-reviewed: Yes
Database(s):
DOI: 10.1016/j.is.2009.04.002.
Google Scholar cites: Citations
Link(s): Paper link
Added by Wikilit team: Added on initial load
Search
Article: Google Scholar BASE PubMed
Other scholarly wikis: AcaWiki Brede Wiki WikiPapers
Web search: Bing Google Yahoo!Google PDF
Other:
Services
Format: BibTeX
Requirements-oriented methodology for evaluating ontologies is a publication by Jonathan Yu, James A. Thom, Audrey Tam.


[edit] Abstract

Many applications benefit from the use of a suitable ontology but it can be difficult to determine which ontology is best suited to a particular application. Although ontology evaluation techniques are improving as more measures and methodologies are proposed, the literature contains few specific examples of cohesive evaluation activity that links ontologies, applications and their requirements, and measures and methodologies. In this paper, we present ROMEO, a requirements-oriented methodology for evaluating ontologies, and apply it to the task of evaluating the suitability of some general ontologies (variants of sub-domains of the Wikipedia category structure) for supporting browsing in Wikipedia. The ROMEO methodology identifies requirements that an ontology must satisfy, and maps these requirements to evaluation measures. We validate part of this mapping with a task-based evaluation method involving users, and report on our findings from this user study.

[edit] Research questions

"the literature contains few specific examples of cohesive evaluation activity that links ontologies, applications and their requirements, and measures and methodologies. In this paper, we present ROMEO, a requirements-oriented methodology for evaluating ontologies, and apply it to the task of evaluating the suitability of some general ontologies (variants of sub-domains of the Wikipedia category structure) for supporting browsing in Wikipedia."

Research details

Topics: Ontology building [edit item]
Domains: Linguistics [edit item]
Theory type: Design and action [edit item]
Wikipedia coverage: Sample data [edit item]
Theories: "Undetermined" [edit item]
Research design: Experiment, Statistical analysis [edit item]
Data source: Experiment responses, Wikipedia pages [edit item]
Collected data time dimension: Cross-sectional [edit item]
Unit of analysis: Category [edit item]
Wikipedia data extraction: Live Wikipedia [edit item]
Wikipedia page type: Policy, Information categorization and navigation [edit item]
Wikipedia language: English [edit item]

[edit] Conclusion

"Following the ROMEO methodology, we identified requirements that an ontology must satisfy in order to support browsing in Wikipedia, and mapped these requirements to evaluation measures. We validated part of this mapping by conducting a task-based user study that compared variants of two sub-domains of the Wikipedia category structure; variants were obtained by untangling the original (tangled) sub-domains. We also experimented with a technique for generating a variant ontology by clustering categories, but the resulting ontologies were not comparable with the original Wikipedia sub-domains. The user study confirmed that tangledness might be desirable in ontologies and category structures that support browsing in general knowledge application areas like Wikipedia; this is especially significant for tasks that require the user to locate specific information."

[edit] Comments


Further notes[edit]

Facts about "Requirements-oriented methodology for evaluating ontologies"RDF feed
AbstractMany applications benefit from the use of Many applications benefit from the use of a suitable ontology but it can be difficult to determine which ontology is best suited to a particular application. Although ontology evaluation techniques are improving as more measures and methodologies are proposed, the literature contains few specific examples of cohesive evaluation activity that links ontologies, applications and their requirements, and measures and methodologies. In this paper, we present ROMEO, a requirements-oriented methodology for evaluating ontologies, and apply it to the task of evaluating the suitability of some general ontologies (variants of sub-domains of the Wikipedia category structure) for supporting browsing in Wikipedia. The ROMEO methodology identifies requirements that an ontology must satisfy, and maps these requirements to evaluation measures. We validate part of this mapping with a task-based evaluation method involving users, and report on our findings from this user study.port on our findings from this user study.
Added by wikilit teamAdded on initial load +
Collected data time dimensionCross-sectional +
ConclusionFollowing the ROMEO methodology, we identiFollowing the ROMEO methodology, we identified requirements that an ontology must satisfy in order to support browsing in Wikipedia, and mapped these requirements to evaluation measures. We validated part of this mapping by conducting a task-based user study that compared variants of two sub-domains of the Wikipedia category structure; variants were obtained by untangling the original (tangled) sub-domains. We also experimented with a technique for generating a variant ontology by clustering categories, but the resulting ontologies were not comparable with the original Wikipedia sub-domains. The user study confirmed that tangledness might be desirable in ontologies and category structures that support browsing in general knowledge application areas like Wikipedia; this is especially significant for tasks that require the user to locate specific information.e the user to locate specific information.
Data sourceExperiment responses + and Wikipedia pages +
Doi10.1016/j.is.2009.04.002 +
Google scholar urlhttp://scholar.google.com/scholar?ie=UTF-8&q=%22Requirements-oriented%2Bmethodology%2Bfor%2Bevaluating%2Bontologies%22 +
Has authorJonathan Yu +, James A. Thom + and Audrey Tam +
Has domainLinguistics +
Has topicOntology building +
Issue8 +
Pages686-711 +
Peer reviewedYes +
Publication typeJournal article +
Published inInformation Systems +
Research designExperiment + and Statistical analysis +
Research questionsthe literature contains few specific exampthe literature contains few specific examples of cohesive evaluation activity that links ontologies, applications and their requirements, and measures and methodologies. In this paper, we present ROMEO, a requirements-oriented methodology for evaluating ontologies, and apply it to the task of evaluating the suitability of some general ontologies (variants of sub-domains of the Wikipedia category structure) for supporting browsing in Wikipedia.ure) for supporting browsing in Wikipedia.
Revid10,928 +
TheoriesUndetermined
Theory typeDesign and action +
TitleRequirements-oriented methodology for evaluating ontologies
Unit of analysisCategory +
Urlhttp://dx.doi.org/10.1016/j.is.2009.04.002 +
Volume34 +
Wikipedia coverageSample data +
Wikipedia data extractionLive Wikipedia +
Wikipedia languageEnglish +
Wikipedia page typePolicy + and Information categorization and navigation +
Year2009 +