Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces
Abstract & Details
Research Area
Computer Engineering
Keywords
: Deep web
two-stage crawler
feature selection
ranking
adaptive learning
Abstract
As wide area of web grows at a very fast pace, there has been increased interest in techniques that help efficiently locate wide web interfaces. However, due to the large volume of web resources and the dynamic nature of deep web, achieving large coverage and high efficiency is a challenging issue. We propose a two-stage framework, namely Smart Crawler, for efficient harvesting wide web interfaces. In the first stage, It is site based searching for center pages with the help of search engines, it avoid to visit large number of pages. To achieve more accurate results for a focused crawl, It is ranking the websites to prioritize highly relevant ones for a given topic. In the second step, It searches fast in-site searching by extracting most relevant links with an adaptive link-ranking. To eliminate bias on visiting some it also contain highly relevant
links in hidden web directories, we design a link tree data structure to achieve wider coverage for a website.
License
This work is licensed under a Creative
Commons
Attribution-ShareAlike 4.0 International License.
Author Information
| # | Name | Institute / Affiliation |
|---|---|---|
| 1 | Mr. Narawade Akshay R. | Shree. Saibaba Institute of Engineering Research and Allied Sciences College, Rahata. |
| 2 | Mr. Pawar Dinesh V. | Shree. Saibaba Institute of Engineering Research and Allied Sciences College, Rahata. |
| 3 | Mr. Sulane Kartik S. | Shree. Saibaba Institute of Engineering Research and Allied Sciences College, Rahata. |
| 4 | Mr. Cholke Dnyaneshwar R. | Shree. Saibaba Institute of Engineering Research and Allied Sciences College, Rahata. |
| 5 | Prof. Dange P.A | Shree. Saibaba Institute of Engineering Research and Allied Sciences College, Rahata. |
How to Cite
Use the following formats to cite this article in your research.
APA Style
R., Mr. Narawade Akshay, V., Mr. Pawar Dinesh, S., Mr. Sulane Kartik, R., Mr. Cholke Dnyaneshwar, & P.A, Prof. Dange (2016). Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces. International Journal of Advance Research and Innovative Ideas In Education, 2(2), 207-210.
MLA Style
R., Mr. Narawade Akshay, et al. "Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces." International Journal of Advance Research and Innovative Ideas In Education, vol. 2, no. 2, 2016, pp. 207-210.
IEEE Style
Mr. Narawade Akshay R., Mr. Pawar Dinesh V., Mr. Sulane Kartik S., Mr. Cholke Dnyaneshwar R., and Prof. Dange P.A, "Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces," International Journal of Advance Research and Innovative Ideas In Education, vol. 2, no. 2, pp. 207-210, 2016.
Vancouver Style
R. Mr. Narawade Akshay, V. Mr. Pawar Dinesh, S. Mr. Sulane Kartik, R. Mr. Cholke Dnyaneshwar, P.A Prof. Dange. Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces. International Journal of Advance Research and Innovative Ideas In Education. 2016;2(2):207-210.
Harvard Style
R., Mr. Narawade Akshay, V., Mr. Pawar Dinesh, S., Mr. Sulane Kartik, R., Mr. Cholke Dnyaneshwar, & P.A, Prof. Dange (2016) 'Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces', International Journal of Advance Research and Innovative Ideas In Education, 2(2), pp. 207-210.
Chicago Style
R., Mr. Narawade Akshay, et al. "Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces." International Journal of Advance Research and Innovative Ideas In Education 2, no. 2 (2016): 207-210.
Turabian Style
R., Mr. Narawade Akshay, et al. "Smart Crawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web Interfaces." International Journal of Advance Research and Innovative Ideas In Education 2, no. 2 (2016): 207-210.
Related Research
A Brief Review of Performance Evaluation of Global and Local Image Thresholding Based on Computational Efficiency
Download PDF
Performance Evaluation of Global and Local Image Thresholding Based on Computational Efficiency
Download PDF
Enhancing Handwritten Alpha-Numeral Recognition Using GAN-Based Synthetic Data Generation
Download PDF
Comprehensive Review of Existing Chatbot Systems for Career Assistance, Resume Support, and ATS-Aware Guidance
Download PDF
Development of an AI-Powered Multimodal Web Assistant with Intelligent Resume Building and ATS Enhancement
Download PDF
A Deep Learning-Based Framework for Mood-Oriented Music Recommendation Using Facial Expression Analysis
Download PDF