RSS feed for this section

Archive | General

Determining Bias to Search Engines from Robots.txt

This piece forms part of our academic series of research pieces relating to search engine optimization. It is authored by Yang Sun, Ziming Zhuang, Isaac G. Councill, and C. Lee Giles (2007). All copyrights belong to the respective authors. Abstract: Search engines largely rely on robots (i.e., crawlers or spiders) to collect information from the Web. […]

Read More Comments Off on Determining Bias to Search Engines from Robots.txt