CRAWLER Introduction

CRAWLER is a technology set for the acquisition of data from the Internet or other source and for the operation of fulltext, context and exact search over the data.

Acquiring data in CRAWLER

CRAWLER acquisition technology implements the HTTP(S) protocol and is able to effectively communicate with tens of thousands remote servers at once through one instance (through one server). The performance of CRAWLER acquisition technology thus allows to put together a high-quality text copy of the average national Internet in 24-hour interval. For continuous operation, CRAWLER also offer optimizing technology (evaluation of source variability) and detailed monitoring.

CRAWLER for fulltext search

CRAWLER fulltext search technology works with the approximate match of occurrences of key words based on which it stipulates the relevance (order) of all known documents. This way is in particular convenient for hand search and is thus characteristic for search engines.

CRAWLER for context search

CRAWLER context search technology works with the similarity of texts based on which it stipulates the relevance (order) of all known documents or their parts. This way is also convenient for manual and automatic searches.