Comments on: How to Block Java user-agents

By: Fighting Bots Via Their Bad Requests - PlanetMike's Technology Journal

Fighting Bots Via Their Bad Requests - PlanetMike's Technology Journal — Sun, 07 Jun 2009 13:46:11 +0000

[...] access my web site? I’ve also decided to block access to my web site by Java user agents. See How To Block Java User-Agents for someone else’s similar approach to the Java [...]

By: Corey

Corey — Thu, 26 Feb 2009 04:27:09 +0000

Yuri, rewrite rules can be implemented to block IP address ranges:

RewriteCond %{REMOTE_ADDR} 213\.93\.196\.\d\d?\d? RewriteCond %{HTTP_USER_AGENT} Java.* RewriteRule ^/(.*)$ /$1 [F]

\d represents a single digit in regular expressions, and a question mark ? makes that character optional

By: Yuri

Yuri — Tue, 24 Feb 2009 13:02:20 +0000

Thanks for Java-bot explanations. Recently I too have found such and other robots on a site.
Many robots steal contents of pages of a site, so the decision to block them is correct.

But I do it a little in another way, because
the Rewrite Enginerules rules is not convenient for blocking of ranges of IP-addresses, therefore it does a script on PHP, likely:
$block= array(
“84.120.0.0-84.123.255.255″,
“122.198.0.0-122.198.255.255″,
“205.209.128.0-205.209.191.255″
);
function checkIP($ip) {
for ($i=0; $i= $b_IP && $IP <= $e_IP) return true;
}
return false;
}

“Manually” blocking IP and UserAgent is not the best practice, so I use robots detection by pseudo-picture loading and JavaScrips evaluating. But Java-bots loaded all pseudo-pictures and evaluate JavaScrips!
One way to detect Java-bots – by UserAgent’s field, but it is not so difficult тo change this fieled.
What to do in this case?