Ok, so this is going to be another post where I rant, but I’ll keep it short. A client called me to tell me that she thought her server was having issues and was acting strange. They were experiencing dramatically high hits to their website in what was supposed to be a down time for them, so I told her that I would take a look at the apache access log to see if there was anything strange. This particular client runs a dedicated server and does all of their server admin themselves (except for when they need me).
When I opened the log folder I think my jaw hit the keyboard - there was a 980MB access_log file in there! Almost a gig of logs. Since this client doesn’t run any statistical software, I was forced to download the file and process it with my local version of AWStats. It took 3 hours to download. Aaaahhhh. For the love of God, rotate your logs!!
Anyway, here are two handy links to learn how to process logs on your local computer with AWStats or a text editor.
- AWStats Mini Tutorial - AWStats can be kind of intimidating for those that don’t know perl (php’s super geek cousin), but this tutorial will get you everything that you need. You’ll need to have apache set up on your machine, and there is a tutorial there for that as well.
- Max Splitter - here is the tool I used to split that huge file into manageable chunks. Notepad++ can handle some pretty large files, but it crashed on this one.
Why do you need both?
Well it’s easier to just look at the log file to see if someone is trying to hack your site. It’s pretty obvious when one ip is hogging the server (as was the case in this instance). In the past I’ve also spotted XSS hacking attempts by looking at the log…it’s pretty obvious when some script kiddie is trying to hack your site.
AWStats will help you once you think you’ve found the ip that is causing the problem. You can filter the visitors list by ip and see stats on just that IP. In my case it showed that the IP in question had use as much bandwidth in the last day and a half as 430,000 visitors had used in the last 2 months. I wonder why their site was slow?
