I am experiencing high traffic on my website at certain times of the data, at regular intervals. I am trying to determine whether GoogleBot or BingBot access of my website correlates with the high load we experience on our server.
Is there a log analyzer or any other way by which I can trace googlebot's access of my website - and the time it occurred and visually show that to me, so I can connect the two metrics (googlebot access vs high load on server)?
Details of what happens during the load is described here: https://dba.stackexchange.com/questions/123946/high-data-traffic-from-mysql-server-to-web-server
This is a Magento website with a large number of products.