Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I'm working on a site with a large music content database, and recently some app developer looking for a bit of a reputation boost has launched an app that completely piggybacks on our database: a user will search for a track in the app, the app will send the request to our search page and scrape the results, returning the top result to the user. Here are some things I've tried:

  • Returning 444 for the app's user agent (but they changed it to a legit browser string).
  • Detecting a referrer on the search page, and returning 404 if no referrer is found (but it would be easy enough for them to spoof a referrer).

One idea I thought of is some sort of token sent along with the search query any page with the search box, and the search result validates this token and, if invalid, returns 404. Are there any tried-and-true techniques that do this kind of thing? Or anything else I can do to prevent this data thief? He's completely unapologetically stealing our data and crippling our site!

By the way I'm using Django, in case there's anything built in to there that could help me.

share|improve this question
    
Your token idea would work if you generated a unique number which you stored in the database temporarily. You are obviously accessing the database for the search results so the overhead would be next to nothing, only really the call to generate it. All databases I know have built-in methods to generate random GUIDs –  Geezer68 Jan 23 at 12:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.