Baning IP(s)

[chance]

I run Abyss on a home w2k pro machine serving up about 60 visitors/350 hits a day. Even though small I have a good positioning for most of my pages on Google and the other major search engines which leads to my problem.

Most robots are considerate and respect the robots.txt file. Two bots, however, continually crawl pages without even reading the robtos.txt file which is set to exclude them. These are grub-clients and various incarnations of ZyBorg. They eat up bandwidth and for the most part don't send any visitors my way.

What I want to be able to do is Ban these bots by IP address(s), but can't figure out how to do it.

Does anyone know of a program, script or method that can be used with Abyss to ban ips? I don't know anything about php and although people have mentioned firewalls being able to ban ips, I can't see any way in the instructions for Zone Alarm to do it.

Thanks
chance

[EvaStar]

The first thing I would suggest chance, is using php in this pages and writing a script that you can use to block these IP's. I will leed you to www.daydreamgraphics.com which will provide you will tutorials on php. It include a tutorial on doing this. When you are first on the site look to the right under tutorials and php you will have to do a little search through the tutorials from there. This will only work with the pages you deside to include this in.
If there is bandwidth error try hotscripts.com or spoono.com, these are the only ways I have found to block things like this with abyss. I know in apache you can add line to the httpd.conf file to block ip's.
If you have a d-link router you can set it to not allow any activity from specified ip's.
I know of no third party programs to help you with this.

Hope this will help you.

[chance]

Thanks EvaStar. Found a couple of php scripts but that is going to involve a lot of learning.

Also found Analog X Port Mapper that says it has the capability to block IP's, but can't get it to work.

The ZyBorg robot from Looksmart crawls my site about every 2 minutes and according to the log doesn't ever call up the robot.txt file.

[EvaStar]

most robots now don't even look at your meta tags to determine if they are allowed to search or not. But using just the robot.txt file will not work very well.
I suggest that you might want to read the book O-Reily called HTML4.0 it gives alot of info on stopping BOTS.

[chance]

Does the O'Reily cover Windows servers? Most of what I have seen from them is Linux and Apache. I really like the Abyss Server but could sure use an .htaccess type configurable file.

Most of the bots are well behaved and turn off as soon as I update the robots.txt, but not the Looksmart/WISENUT bots, nothing seems to stop them except banning according to what I read on the webmasters world Forum. When I shut down all bots with the robots.txt.:

User-agent:*
Disallow: /

all bot activity (search engine type) Except Zyborg and the various grub-clients ceased. zyborg doesn't even pretend to look at robots.txt, it just pulls up pages. grub-client(s) looks at the robots.txt, ignores the entries, then goes ahead and does its thing anyway. Even slurp obeys the robots file, although it may take a day or two to stop.

[Riplin]

maybe try buying a router that can ban ip ranges i've seen them for around $50-$100

I use one that can ban ip's

[ReptileKing234]

I also am in dire need to banning an IP. My site is very small, (a lot smaller than what chance's site is) but already I am plagued by someone crashing my servers. I will try to use some of the previous suggestions in order to ban the ip of the person believed to be the hacker, but if anyone else has info, or any way that might be more effective on people than bots, it would be appreciated. Just like chance, I have no knowledge of php, and schooling is taking up most of my time. Thanks for any info.
_________________
ReptileKing234
A voice of reason in a world gone mad

[chance]

Thanks all for the suggestions. Am out of town for a week and just wanted to let yu know that I am not ignoring the forum.

Thanks
Chance

[s1asher]

[EvaStar]

Since every one has switched to the topic of firewalls and routers I shall suggest the d-link 604. It has 4 port lan access and 1 wan access. Easy to configure, plug in and play, just setup dhcp support under windows, or dhcpcd for linux. Allows the blocking of IP's from internal access to external and external to internal. Works great for blocking sites you don't want children to access as well as sites you don't want guessed that use yous computer(s) to access. Blocks all but the ports you would use to access webpages with before you configure.
I use as a firewall for my server and my families computers.

[chance]

[chance]

aprelium

I have been thru the forum again and again looking for an idea concerning banning of ip's and seen the numerous refs to .htaccess. When .htaccess is mentioned the response is Users/Groups/Password Protection. While this looks good for allowing individual access to individual files/folders while blocking the rest of the world, is there anyway of blocking one individual ip while letting the rest of the world in?

Thanks
Chance

[Anonymoose]

Instead of using Zonealarm, try using Kerio Personal Firewall - www.kerio.com - it's a *lot* more configurable than Zonealarm's simple "Do you want this to be a server?" crap... You sound like you're techie enough to handle a few more settings :wink:

All it would take is one rule to totally block an IP from accessing your machine. Give me a shout if you decide to install it and I'll point out the rule to set up if you can't figure it out :)

[chance]

[stitch]

Is there a free software firewall program that can be configured to ban specific ip's? I'm using zonealarm now...

*huddling at the monitor for warmth and nibbling on stale bread crusts*

[stitch]

oooohkay... found the free kerio firewall... now how to configure to ban ip?

[Anonymoose]

You want the older V2.xx Kerio, not the 4.xx one currently offered for free on their site - it doesn't have the same gui, makes it very difficult to find options compared to the previous one. Haven't yet found any IP specific options in it, they seem to have gone for the more cuddly ZoneAlarm approach.

Don't have it installed at the moment, but I'll stick it on my laptop and take some screenshots when I get chance.

[Anonymoose]

Hmm nope, found the options in the new one now. Will knock up some screenshots!

[Anonymoose]

Don't have anywhere to put screenshots at the moment... I think I've broken down the description enough to make it simple to work through but if not I can email you the screenshots.

If you have other personal firewall software installed you're going to need to uninstall it to avoid conflicts. I'm going to assume you have some knowledge of how to operate a personal firewall in terms of setting the rest of it up - this is just for banning IP's using Kerio. If you don't, there's a fairly complete help system installed with it to get you started.

1 ) Right click the system tray icon and choose configuration

2 ) Go to the Network Security Tab

3 ) Click Packet Filter

4 ) Click Add

(The Important Bit)

IP Groups are a way of combining all banned IP's into one easy to manage group - you will be able to say Block Banned IP's rather than setting up the whole set of rules individually for however many users you want to ban.

Remember that some users on dialup and DSL/cable may have non static IPs, so blocking a whole group of IP's may be the only way to get rid of them. This means banning the whole subnet of IP's the user might connect from.

For example, if their IP is 10.0.0.3 you would ban 10.0.0.1-10.0.0.254. Do this by selecting Address Range instead of Host in the steps explained below and entering the first and last IP in the range. Only use the range option if you are sure no other users from that range need to connect.

e.g. If you are serving on a LAN and want all LAN users except one IP to be able to access the server, you would use an IP ban, not a range ban. If you are serving over the internet and want to ban one user and don't have logs of any other users from the same range who you still want to allow access to, block the whole range.

5 ) Click IP Groups.

6 ) Click Add and enter a group name such as "Banned Abyss Users". You can leave the description blank or enter extra details here if you want.

7 ) Leave the Type option set to Host and enter the first IP you want to ban here. Click OK.

8 ) For each additional IP you want to ban, click Add, then select the group Banned Users and enter the IP to ban as above. Make sure you select the group banned users or the final steps will not work correctly.

Now you have all the users to ban set up, you are ready to create a banning rule.

9 ) Click Filter Rules, then Add.

10 ) Enter a description for the rule - something appropriate
e.g. Ban Users from Abyss Webserver

11 ) Click Browse and browse to the .exe file for Abyss
e,g C:\Program Files\Abyss Web Server\abyssws.exe on my system.

12 ) Leave the group name as Default, or enter something like Abyss Webserver Rules. All the Group does is group similar rules together, it doesn't affect how the ban will work.

13 ) For protocol, click Add and select TCP, the default.

14 ) If you want to ban the user from your machine completely, not just Abyss, skip this step. Otherwise, in the Local section, click Add and enter the port number your server is running on.

15 ) In the Remote section, click Add, then select IP Group and the group you used to enter the banned users in.

16 ) Select Incoming and Deny from the checkboxes at the bottom of the dialogue.

17 ) If you want a box to pop up when they try to connect, click the "Show Alert to User" box.

18 ) Click OK.

Bingo! You've just blocked all the bad guys from your server :)

To add further banned users in future, follow steps 1-5 and 7. Skip 6.

Hope this helps!

[stitch]

Thanks Anonymoose-- You're the best!

Not only are you super smart but you are really good at explaining things clearly... and I really appreciate your taking the time to go through it step by step!
I have found better support and more useful info here at the Abyss forums than with any other software that I *paid* for...

I'm really happy with the Kerio firewall, too... thanks for letting me know about it! :D