Adaptive indexing for content-based search in P2P systems
|Adaptive indexing for content-based search in P2P systems|
|Authors:||Aoying Zhou, Rong Zhang, Weining Qian, Quang Hieu Vu, Tianming Hu|
|Citation:||Data and Knowledge Engineering 67 (3): 381-398. 2008.|
|Publication type:||Journal article|
|Google Scholar cites:||Citations|
|Added by Wikilit team:||Added on initial load|
|Article:||Google Scholar BASE PubMed|
|Other scholarly wikis:||AcaWiki Brede Wiki WikiPapers|
|Web search:||Bing Google Yahoo! — Google PDF|
One of the major challenges in Peer-to-Peer (P2P) file sharing systems is to support content-based search. Although there have been some proposals to address this challenge, they share the same weakness of using either servers or super-peers to keep global knowledge, which is required to identify importance of terms to avoid popular terms in query processing. As a result, they are not scalable and are prone to the bottleneck problem, which is caused by the high visiting load at the global knowledge maintainers. To that end, in this paper, we propose a novel adaptive indexing approach for content-based search in P2P systems, which can identify importance of terms without keeping global knowledge. Our method is based on an adaptive indexing structure that combines a Chord ring and a balanced tree. The tree is used to aggregate and classify terms adaptively, while the Chord ring is used to index terms of nodes in the tree. Specifically, at each node of the tree, the system classifies terms as either important or unimportant. Important terms, which can distinguish the node from its neighbor nodes, are indexed in the Chord ring. On the other hand, unimportant terms, which are either popular or rare terms, are aggregated to higher level nodes. Such classification enables the system to process queries on the fly without the need for global knowledge. Besides, compared to the methods that index terms separately, term aggregation reduces the indexing cost significantly. Taking advantage of the tree structure, we also develop an efficient search algorithm to tackle the bottleneck problem near the root. Finally, our extensive experiments on both benchmark and Wikipedia datasets validated the effectiveness and efficiency of the proposed method.
"To that end, in this paper, we propose a novel adaptive indexing approach for content-based search in P2P systems, which can identify importance of terms without keeping global knowledge."
|Topics:||Other information retrieval topics|
|Theory type:||Design and action|
|Wikipedia coverage:||Sample data|
|Data source:||Archival records, Experiment responses, Wikipedia pages|
|Collected data time dimension:||Cross-sectional|
|Unit of analysis:||Article|
|Wikipedia data extraction:||Dump|
|Wikipedia page type:||Article|
|Wikipedia language:||Not specified|
"As a result, our system avoids several severe problems caused by maintaining such global knowledge. Nevertheless, it still achieves comparable retrieval efficiency as those systems keeping global knowledge. Although the structure of our system is partially based on a tree structure, our search algorithm guarantees no bottleneck at the root or nodes near the root. In addition, we also introduced several techniques to further improve the system performance. Finally, our extensive experiments demonstrated the effectiveness and efficiency of the proposed approach."
"Wikipedia pages Documents
We evaluated all systems on three datasets. The first one is the benchmark dataset containing four different types of documents: MED, CISI, CACM, and TIMES used by Smart , with 1033, 1460, 3204, 425 documents and 30, 35, 64 and 83 queries, respectively. The two remaining datasets are the 2005 TREC publish spam corpus consisting of 84,053 files obtained from http://plg.uwaterloo.ca/gvcormac/treccorpus/ and the Wikipedia dataset containing 1,000,000 pages downloaded from Wikipedia (http://wikipedia.org), a free multilingual online encyclopedia. Queries of these two datasets are generated randomly from the categories of TREC and Wikipedia."