Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to search the web for sites that are in our industry that use the same Adwords management company, to ensure that the said company is not violating our contract, as they have been accused of doing.

They use a tracking code in the template of every page which has a certain domain in the URL, and I'm wondering if it's possible "Google" the source code using some bot that crawls the code rather than the content?

For example, I bought an unlimited license for an image gallery, and I was asked to type the license number in a comment just before the script. I thought it was just so a human could look at the source and find out if someone paid, but it turned out that it was actually that they had a crawler looking for their source code and that comment. If it ran across the code on your site, it would look for the comment, and if it found one, it would check to see if it was an existing one. If not, it would first notify you of your noncompliance, and then notify the owner of the script.

Edit: I'm looking to index HTML and JavaScript only, not the server-side languages or Java.


migration rejected from stackoverflow.com Feb 16 '14 at 2:36

This question came from our site for professional and enthusiast programmers. Votes, comments, and answers are locked due to the question being closed here, but it may be eligible for editing and reopening on the site where it originated.

closed as off-topic by dan Feb 16 '14 at 2:36

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions asking us to recommend a tool, library or favorite off-site resource are off-topic for Pro Webmasters as they tend to attract opinionated answers and spam." – dan
If this question can be reworded to fit the rules in the help center, please edit the question.

Before anyone posts it, Koders.com is NOT what I'm looking for. That does not index HTML or other markup languages, only open-source scripting languages. It would be awesome if I could find one where I could suggest Google keywords and have it crawl the source code of the top n Google results. – Dexter Jan 6 '11 at 16:26