Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Is there a way to get recorder real network traffic to web server, e.g. from web server logs (Apache), and replay this traffic to either profile web application (in Perl) under real load, or benchmark and compare speed of different implementations before choosing one or the other?

If it matters, webapp is written in Perl, and runs under plain CGI, FastCGI, mod_perl (via ModPerl::Registry), PSGI (via Plack::App::WrapCGI).

Crossposted from StackOverflow

share|improve this question

4 Answers 4

up vote 1 down vote accepted

Depending on the scale, you will need a specific software, though a specific software is recommended for any performance/load/stress tests hence they can provide better results.

For medium traffic, I can recommend Apache Jmeter, or Selenium RC. I think replay Apache HTTP Server (yes, this is the fullname from our famous Apache) logs is easier to do into Apache Jmeter.

share|improve this answer

If you want to record and replay real traffic, I think the most direct solution would be to log full request headers to a file and then use a solution like curl to "replay" the requests against your webserver.

share|improve this answer
    
This can be basically be done with JMeter I recommended above. Plus, it has the options to generate many reports, making easy to keep track of. –  Dave Nov 8 '10 at 11:06
    
@Dave - Above? :) –  danlefree Nov 9 '10 at 4:57

Why would "real" data give you more information than a simulation that scales until the server breaks? At the very least you should be able to amplify the load signal from the load data to give a range on the performance... Also, seems like the log data would only profile the request, not the response rate from the client request, meaning one client request to close the connect super fast, the other might lag.

share|improve this answer
1  
The anegdotical evidence suggests that the web app I want to test is I/O bound, not CPU bound. That's why the need for real-data (or sufficiently similar). –  Jakub Narębski Nov 8 '10 at 0:12
    
Whether you use real data or not might not matter for profiling the execution times of different implementations, but using real data is still useful for determining where your actual application bottlenecks are. This means you don't waste time on micro-optimizations that yield no real-world benefits, and instead can identify useful macro-optimizations that are based on real-world conditions. –  Lèse majesté Nov 8 '10 at 1:57

This may do what you want:

http://patrick.net/sprocket/rwt.html

It has an ultra-simple proxy for recording traffic, and an even simpler perl script for playing it back!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.