log analyzer

Hello, I need a software program that can analyze data from large logs in different formats on different streaming servers. Please react only if you have experience building such log analyzers. The log files are created on different streaming servers and on different machines. The format of the log files can be configured on the current servers we use. Description of the execution for the correct configuration of the server will be a part of the job. At the moment we download the log files using rsync. Some of the servers are: - Helix (real) - Windows Media 9 - Darwin (quicktime) - Kasenna (mpeg2) Because servers can change as the machines, the program has to be built up in a very modular way so it can adapt easily when servers change or new servers are being added. We would like the log files to be analysed like awstats and get easy information of almost anything you would like to know or see interesting relations like where the visitors come from, what files are being used or when there are cpu peaks or: 1- Number of concurrent streaming viewers 2- Amount of concurrent bandwidth used 3- Amount of storage used 4- Detailed viewer stats related to media. 5- Total amount of data streamed In future we would like to have also real time information. Because we would like all different log files together I think it would be appropriate to use one single database where it all get stored and you can analyse all sort of data. My own preference is MySql, but I know it becomes worse when the queries get complicated. I would prefer java too. Other systems are possible too. ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request. 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform unix, linux, apache, windows xp streaming servers: (darwin, helix, windows media)