Determining Bias to Search Engines from Robots.txt
Yang Sun
Ziming Zhuang
Isaac G. Councill
C. Lee Giles