[BBLISA] Re: Large scale log processing

[Sean Lutner]

I'd second the recommendation for splunk. It's a fantastic product, is  
easy to setup and would provide you with a way to aggregate and then  
easily search over all your data. Aggregation is the easy part, the  
searching, correlation, etc is not easy. I've implemented splunk at  
three different places and am in the middle of a deployment currently.

On May 15, 2009, at 9:49 AM, seph wrote:

> Mike Sprague <mfs at komerex.com> writes:
>
>> I work for a web hosting company with about a thousand linux servers.
>> We're discussing options on how to process the logs mainly from our  
>> mail
>> and web servers to make troubleshooting easier.  We're not really
>> looking for long term storage; just a better way to be able to search
>> the logs to diagnose either specific customer issues, broad system
>> attacks, issues across a pool of servers or issues with a specific  
>> server.
>
> splunk would be the obvious commercial product for this space.
>
> http://www.opensyslog.com is a online hosted log consolidation tool.  
> In
> beta.
>
> I've seen a variety of open source things in the log processing  
> world. I
> don't think they're generally very flashy, and I'm not sure what the
> current favorites are. googling around syslog processing finds some
> suggestions.
>
> seph
>
> _______________________________________________
> bblisa mailing list
> bblisa at bblisa.org
> http://www.bblisa.org/mailman/listinfo/bblisa
>