Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I manage a website with about 1,000 articles that need to be searchable by my members.

The site search engines I've tried all had their own problems:

Since it's written in perl, it was a bit hacky to integrate with my PHP-based CMS. I basically had to file_get_contents the search results page. However, FDSE had the best search results.

Ugh, the search results SUCK. It can't find documents even using unique strings. I'm so surprised that a Google search product is this bad. Nor can I get any answers on their 'help' forums, and I am a paying user. Boo, Google. Boo.

Again, bad search results. Unable to locate some phrases used in link text. Better results than Google CSE though. Shame on Google that a free PHP script has better search results than their paid application.

This one looked really promising. I got all set up with their PHP API client. But it would only randomly add articles that I submitted. Out of 700+ articles I pushed to the index through their API, only 8 made it in. Unable to find any help on this subject.

Update for IndexTank -- Got the above issue fixed, so this looks most promising so far.

The site itself runs on php/mysql and FreeBSD, though this shouldn't matter for a web crawling indexer.

I've looked at Lucene, but I don't know anything about Java or installing Java programs on my web server.

I also do not have root access on my web server, if this would be required for installation.

I really don't need a lot of fancy features. It just needs to be able to crawl my web site and return great (even decent!) search results. I don't need any crazy search operators. It doesn't need to index off my primary domain. It just needs to work!

Thanks, Hive Mind!

share|improve this question

closed as off-topic by John Conde Oct 22 '13 at 20:42

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions asking us to recommend a tool, library or favorite off-site resource are off-topic for Pro Webmasters as they tend to attract opinionated answers and spam." – John Conde
If this question can be reworded to fit the rules in the help center, please edit the question.

Mods - I would add the tag 'site-search' to this post, but it's a new tag and I don't have the rep. – Ian Nov 17 '10 at 21:28
done ` ` ` ` – Mark Henderson Nov 17 '10 at 21:36
I've been pleased with the CSE results, but I do think the Google's Help Forums are lacking. – plntxt Nov 17 '10 at 21:52
Just a note to say that I'm getting most of my issues resolved with IndexTank, and it's looking like they will be able to get me where I need to be. – Ian Nov 18 '10 at 22:00
Another note to say that, since Indextank got bought by LinkedIn, their status as a search service I could rely on has become unknown. I looked into installing Sphinx, and had little trouble getting exactly what I needed from it. I highly recommend trying out Sphinx. – Ian Dec 10 '11 at 7:19
up vote 2 down vote accepted

I use sphinx. It is built on Lucene as is Solr so it's got all the powerful options you could ever want, but also has plugins for popular open source cms' such as wordpress or drupal to get you going. There are also plugins for popular frameworks such as RoR or symfony if you've built the site using one of those.

share|improve this answer
Sphinx is not built on Lucene. – Mauricio Scheffer Feb 10 '11 at 20:13
True, Sphinx is not built on Lucene. But it does work great, and I was able to (a year later) find a long-term, in-house search solution with Sphinx that's pretty easy to tweak and maintain. I definitely recommend Sphinx. – Ian Dec 10 '11 at 7:22

Solr, which runs on Lucene is terrific.

It's not the easiest thing to get going, but there is a very good book I recommend. I wish I had it before starting out with it:


share|improve this answer
This may be a bit more than I want to deal with. – Ian Nov 18 '10 at 3:44
+1. SolR has a learning curve, but it is amazingly powerful. Wouldn't think of using anything else for my sites. – Kris Nov 18 '10 at 9:13
I would have to install Tomcat and all that as well, right? Is that much of a daunting prospect? Can it be done without root access? – Ian Nov 18 '10 at 19:29

Try using SearchBlox, it is meant for site search and it is free to use. You can create an HTTP collection to crawl through the 1000 pages and make that searchable to your members upon login. Additionally there is a jquery instant search plugin with autocomplete that could also make things better for your members. http://www.searchblox.com/instant-search-jquery-plugin-for-searchblox

share|improve this answer
Hi tss. This reads a lot like a spam post - if you have more details to add, please do so. – Mark Henderson Feb 7 '11 at 0:55
SearchBlox, also built on Lucene, so same problems (for me) as the other Lucene solutions above. – Ian Feb 10 '11 at 4:43
@Ian - SearchBlox is a ready to deploy Lucene based search application and there are a ton of customers using it so when you say it has the same problems as Lucene, can you elaborate on the issues you are facing? – tss Sep 20 '12 at 15:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.