2

I need to log all requests coming into the server. Typical things you would get from Apache log formatting or IIS.

I am using a WSGI standards based server, Apache mod_wsgi, gunicorn, Tornado. It doesn't matter.

Which is an efficient logging method for a production application? My application is in Flask, and I have Python logging set up. Works fine. But I am afraid it will slow my application to a crawl, and it is very difficult to get things like bytes send and received and the rest of granular type logging a server provides. It shouldn't matter the language or application. I think this is applicable to anything a web server can respond to.

Am I correct that I shouldn't fight with a Python framework and let the server do the work?

With something like IIS you have ODBC logging. You don't seem to have that anywhere else.

Improve this question

1 Answer 1

Reset to default
1

Your HTTP server should handle your access logging.

I add additional logging from my application because there are fields that I want log that the HTTP server doesn't have access too:

  • Response category (page, html fragment, image, js, css, json, etc)
  • Page template (name of template file used to generate the page)
  • Logged in user id (HTTP server can log basic authentication user, but most web apps use other types of logins)
  • Session id
  • Campaign info (how the user originally got to your site)

Logging does not usually put undue burden on your servers as long as your are logging only a few lines per request. Logging does need to be synchronized or have one log file per thread. A good logging framework will handle that for you.

Just like with access logs, you need to periodically rotate any logs that your application creates to keep disks from filling up.

Improve this answer
4
  • I believe I'm going to use a script to parse my Apache logs and put them in a database, maybe use cron. I can pipe out to a script but I am afraid it will slow things down. I don' t know.
    – johnny
    Commented Apr 10, 2015 at 15:06
  • Log processing can be very compute intensive and a big project to implement. You can always offload the logs to another machine for processing. You should also consider using Google Analytics which can give you many of the answers you want with much less development work.
    – Stephen Ostermiller ♦
    Commented Apr 10, 2015 at 15:20
  • It is all in-house. I didn't think I could do GA internally.
    – johnny
    Commented Apr 10, 2015 at 15:23
  • It should still work. The JavaScript on the in house pages just need to be able to call out to Google's servers.
    – Stephen Ostermiller ♦
    Commented Apr 10, 2015 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.