I have been using Awstats for many years to process my log files. But I am missing many possibilities (like cross-domain reports) and I hate being stuck with extra fields I created years ago. Anyway, I am not going to continue to use this script.

Is there a modern apache logs analytics solution based on modern storage technologies like NoSQL or at least somehow ready to cope with large datasets efficiently?

I am primarily looking for something that generates nice sortable and searchable outputs with the focus on web analytics, before having to write my own frontends. (so graylog2 is not an option)

This question is purely about log file based solutions.


Splunk can import Apache logs http://docs.splunk.com/Documentation/Splunk/4.3.2/Data/Apachelogslocal

Or review this answer on Stack Overflow

A relatively recent option is to use Flume to collect the logs and use the MongoDB sink plugin for Flume to write the events to MongoDB.

  • Thank you for this answer, I will upvote it, as soon as I have gained the required reputation. I will check it out, but depending on how Splunk calculates the indexing volume, the 500MB/day of the free version will not be sufficient and the pricing seems rather steep for this purpose. – Martin Jun 13 '12 at 10:36

In case someone else comes along this problem: My solution for the moment was to use Goaccess on the log files or on subsets using cat and grep. It may seem as a regression compared to awstats or any software with some result storage, but its performance is great for ad-hoc log file analysis.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.