Is there a way to get recorder real network traffic to web server, e.g. from web server logs (Apache), and replay this traffic to either profile web application (in Perl) under real load, or benchmark and compare speed of different implementations before choosing one or the other?

If it matters, webapp is written in Perl, and runs under plain CGI, FastCGI, mod_perl (via ModPerl::Registry), PSGI (via Plack::App::WrapCGI).

Crossposted from StackOverflow


Depending on the scale, you will need a specific software, though a specific software is recommended for any performance/load/stress tests hence they can provide better results.

For medium traffic, I can recommend Apache Jmeter, or Selenium RC. I think replay Apache HTTP Server (yes, this is the fullname from our famous Apache) logs is easier to do into Apache Jmeter.

| improve this answer | |

This may do what you want:


It has an ultra-simple proxy for recording traffic, and an even simpler perl script for playing it back!

| improve this answer | |
  • Seems like this also can replay ALL traffic, not just GETs like most of the aforementioned. Creating data using POST or PUT might just as well be part of the load test. – oligofren Oct 7 '14 at 10:47

If you want to record and replay real traffic, I think the most direct solution would be to log full request headers to a file and then use a solution like curl to "replay" the requests against your webserver.

| improve this answer | |
  • This can be basically be done with JMeter I recommended above. Plus, it has the options to generate many reports, making easy to keep track of. – Davis Peixoto Nov 8 '10 at 11:06
  • @Dave - Above? :) – danlefree Nov 9 '10 at 4:57

Why would "real" data give you more information than a simulation that scales until the server breaks? At the very least you should be able to amplify the load signal from the load data to give a range on the performance... Also, seems like the log data would only profile the request, not the response rate from the client request, meaning one client request to close the connect super fast, the other might lag.

| improve this answer | |
  • 1
    The anegdotical evidence suggests that the web app I want to test is I/O bound, not CPU bound. That's why the need for real-data (or sufficiently similar). – Jakub Narębski Nov 8 '10 at 0:12
  • Whether you use real data or not might not matter for profiling the execution times of different implementations, but using real data is still useful for determining where your actual application bottlenecks are. This means you don't waste time on micro-optimizations that yield no real-world benefits, and instead can identify useful macro-optimizations that are based on real-world conditions. – Lèse majesté Nov 8 '10 at 1:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.