Blacklisting via Ionic’s Isapi Rewrite Filter
In IIS, banning IP addresses from accessing a website is fairly easy. I rarely do this, however, because I prefer to use a combination of an IP address and a user agent string to identify bad bots that are likely scraping my content or attempting to harvest email addresses.
I try to avoid blocking an IP address at all costs. IP addresses can be forged and changed, so I prefer to rely on an IP address and user agent string combination to identify the culprit that I want to exile. This approach is not fool proof, but I find it be much more reliable.
Scalability is also an issue. The use of an ISAPI filter to process requests for every website on the server or a single file sure makes life easy. The Microsoft IIS configuration console is a mouse-click nightmare on a server with a couple hundred websites.
I use Ionic’s Isapi Rewrite Filter to change the URL structure of websites to be more search engine friendly. This filter uses the PCRE library, and the use of regular expressions is always a huge plus. The rewriting rules are maintained inside one .ini file, so tweaks and updates are a breeze.
Here is an Ionic’s rewrite rule that will let you block access to every site on your server based upon an IP address and user agent string match. In this particular case, I am blocking an email address harvester with IP 24.132.226.94 and user agent Java/1.6.0-oem.
RewriteCond %{REMOTE_ADDR} 24.132.226.94
RewriteCond %{HTTP_USER_AGENT} Java/1.6.0-oem
RewriteRule ^/(.*)$ /$1 [F]
The two conditions on this match use server variables to match the user’s IP address and user agent string to an expression match. The final line is the rewrite rule that matches any file on any website. The [F] flag tells the Ionic’s filter to return an appropriate HTTP status code of 403 Forbidden.
Regular expressions provide the capability to block a range of IP addresses and partial user agent matches. If i wanted to match on any version of this Java-based robot, I could expand the second condition to something like this:
RewriteCond %{HTTP_USER_AGENT} Java/\d.\S*
Similarily, wildcard matches on IP addresses can be used to block ranges of IPs instead of a single address.
The Microsoft vs *NIX server debate will never die. I use both everyday, and I find that the biggest advantage that the open source server environment has over Microsoft is the interface. Using the Ionic’s ISAPI filter allows me to control the URL structure and blacklist for all of my websites easily and efficiently.
I see this method of blocking IPs or blacklisting bots based on IP address and user agent as a great way to simulate an .htaccess approach to the same problem on a Microsoft server.
Great, great.
I’ve got a question if i can : are you the owner of this website ?
Don’t worry, i am asking you this question because i can see Apache/1.3.34 runing on this website and i’m searching to fake the server name for security reason.
I know ServerMask who can do that but it don’t work under iis7…
Sorry for my “apparte”…but if you can anwser, it could be great ;)
Thanks
Yes, this is my site, and no, there are no tricks going on here. I am a huge Wordpress fan, so any blogs I setup are running on Apache.
The Microsoft servers I use are still running IIS 6. Obviously, I am curious about 7’s new URL Rewriting feature, but I hesitate before upgrades like that. Yesterday, I tried to update an XP machine to SP3 and it blue screened during the install.
IIRF can rewrite headers including the “Server:” header that is returned to the requesting client (or browser). IIRF can change the name of the server name, based on the incoming IP, the incoming user agent, the URL, whatever.
Check it out. http://www.codeplex.com/IIRF
As a webmaster, you definitely should use user-agent headers to manager server traffic. But understand that this is purely a pragmatic tactic and not a serious security measure.
I wrote more about this here:
Webmaster Tips: Blocking Selected User-Agents
http://faseidl.com/public/item/213126
Thanks for your thoughts, Andy. I have thought about Guyty’s comment a lot since he inquired about faking the server header, and it is an interesting approach to security.
Most of the attempts to compromise my sites are input form SQL injection. These attacks can easily be automated, so I am not sure masking the type of server will divert them.
I agree that managing traffic is essential to maintaining websites, and since you shared a post I enjoyed I will return the favor. I hope you are not offended by a few bad words.
Impact On Your Bandwidth Will Be Minimal My Ass
http://incredibill.blogspot.com/2008/05/impact-on-your-bandwidth-will-be.html
i’m searching “iirf ip list” and came to this post.
I think this solution is good for just a few IPs.
My case is that there are about 200 IPs as IP allow list, all other IPs are banned to visit the IIS 6.0. don’t know if there is a good solution in iirf regarding this case, or other ways. thanks any way.