1

Given a URL with a CDN domain (e.g. a1856.g2.akamai.net), is there a way to find out the original webpage/service that was requested when it was delivered via that CDN?

I am monitoring my internet usage and the monitoring software spits out URLs visited. Some of these are obvious while others delivered via CDNs are not.

e.g.

  1. www.xkcd.com - the user obviously visited the XKCD website
  2. a1856.g2.akamai.net - I have no idea, this is what I would like to know more about.
  • I don't have a definite answer for you but I don't think this is possible. – Alex Reid Oct 7 '15 at 6:28
3

Short answer

You can't in general. Many "original domains" could be mapped to the same CDN host. In a sense if all you've got is the information you mentioned in your question, it's not enough and "too late" to do anything.

Longer answer

There's some hope if the traffic is in clear. The point is: a CDN is often use to serve "heavy" traffic, while lightweight "initiating" traffic is done through a regular domain and that one mentions the CDN urls.

For example, in the case of browsing activity (in clear or decipherable), you might get away with some tool that parses sniffed web pages to extract links and do the correlation. The principle is: if that CDN was called through some URL, probably some webpage served by the original domain mentioned that exact URL. It will not work for computed URLs via JavaScript etc if the parser does not execute JavaScript, but when it finds a correlation it is most certainly correct.

This may not always work. For example, for at least ten years some software has been downloading their updates through a CDN. In that case there may be no prior webpage to do the correlation (though the software may calls home domain to get the CDN URL, in which case the solution above may catch it).

The assumption of web pages may be weakened by just looking for anything that looks like a URL in traffic instead of assuming well-formed HTML pages. Some TCP traffic that is not web but still in clear would let you link the CDN URL back to the original domain.

If the traffic is encrypted, that kind of information is hidden. Some tools might still find some weaker correlation, like "what domain was requested by same client IP prior to the CDN domain" but such correlation won't be certain.

All in all, you need more information than just "URLs visited".

Tools? Try Justsniffer

I haven't used any tool but one might start with Justniffer.

For http traffic it produces Apache-like log with referer field (see Examples). Here, a query to google.com refered to another URL starting with /csi:

192.168.2.2 - - [15/Apr/2009:17:20:18 +0200] "GET /csi?v=3&s=web&action=&tran=undefined&ei=MvvlSdjOEciRsAbY0rGpCw&e=19592,20292&rt=prt.175,xjs.557,ol.558 HTTP/1.1" 204 0 "http://www.google.it/search?q=subversion+tagging&ie=utf-8&oe=utf-8&aq=t&rls=com.ubuntu:en-US:unofficial&client=firefox-a" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8 Gecko/2009032711 Ubuntu/8.10 (intrepid) Firefox/3.0.8)"

So, this tool can correlate between two queries. Looks like it has potential to do what you need.

Moreover, Justniffer:

It is extensible

Can be extended by external scripts. A python script has been developed to recover all files sent via HTTP (images, text, html, javascript, etc.).

Look at justniffer-grab-http-traffic

An example written in python is the http_parser.py It stores the transferred contents in an output directory separated by domains.

To me, this strongly suggests that starting with sample scripts bundled with justsniffer, it should be relatively easy to write an extension script implementing the suggestions in the "longer answer" above.

That would probably address your need. Else look for sentences or keywords in its description that might hint you at what to type in a search engine to find similar tools.

| improve this answer | |
  • That was helpful in understanding the situation, thanks! As per your suggestion: do you know of tools that parse packets to determine CDN links that follow? And I agree, it's not just web pages I'm after but traffic in general (e.g. a runaway download or software update chewing data). I'd like to know the domain name so I know what to look for on the suspect PC. – Islay Nov 7 '15 at 23:52
  • @Abhi you're welcome. I've updated the answer. Feel free to come back to this question and share what you find. – Stéphane Gourichon Nov 8 '15 at 9:09
0

What is the problem with the second URL? It is the https://www.akamai.com/, well known CDN service. If your really really want to determine the CDN url, use this cdn recognition service. They offer a bookmarklet too

| improve this answer | |
  • 1
    Problem with the second URL: I cannot determine which website was being served using that CDN link. [I doubt a home user was transferring hundreds of MBs to and from Akamai purposefully.] I want to do the reverse of that CDN finder tool i.e. given a CDN url, find out which website it points to. – Islay Oct 7 '15 at 19:57
  • you should monitor your network traffic through an incognito window: many browser extensions use different cdns. – Evgeniy Oct 9 '15 at 7:09
  • Sorry, I don't follow. I'm not monitoring via a browser, but a packet sniffer application which sniffs all network traffic on my LAN. – Islay Oct 10 '15 at 7:57
  • @Abhi if your browser is on while you sniff, it could happen, that any browser extension connects to any cdn – Evgeniy Oct 12 '15 at 7:06
  • 1
    this has nothing to do with a browser on the monitoring machine. This machine only runs a sniffer application which monitors all inbound and outbound internet traffic from the LAN. That traffic, of course, includes web traffic from other people's browsers (incl. any of their extensions). Regardless, this point is irrelevant. I am only interested in knowing if it is possible to find out which original website/service was requested given the CDN address through which it was served. – Islay Oct 12 '15 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.