Frequently Asked Questions
About Web Search Builder

How do you get those key phrases?

Since 2001, we've been developing textual analysis tools to help improve knowledge discovery within our unique collection of 4000+ reports of the National Academy of Sciences, the Institute of Medicine, the National Academy of Engineering, and the National Research Council. Algorithmic key phrase weighting was a necessary precondition, so we developed an approach that takes into account word frequency, phrase frequency, term placement, co-occurrence with other significant terms, containing-sentence significance, and other factors, to give individual phrases a "significance weight" within the context of the chapter or article. This allows us to extract the top N phrases for a given chunk of ASCII text, which we can apply to chapter skimming, reference finding, and search building.

But don't you want to keep me on your site?

The National Academies Press, as part of the National Academies, encourages knowledge exploration, research, and refinement. The language of our reports -- which tend to be distillations of the best experts in a given field -- also tends to include a high proportion of "terms of art," which can be very useful for locating, within the surfeit of Web content, high-quality resources. "Keeping you on the site" is a tired tactic for commercial Websites -- we believe that if these tools are useful, you'll return to take advantage of them, and perhaps, eventually, find a book you want to buy in one form or another.

What else do you do with those phrases?

We have three other primary applications: a) as an adjunct to fulltext searching, in the backend of our book search engine; b) as the key element in "Chapter Skimming," we we can identify, on every chapter page, the two-sentence chunk containing the highest value of high-value phrases, providing a proxy for skimming, and c) as the key backend system for building the Reference Finder.

Is this open source? Where can I get this software?

Currently, you can't -- it has been written in Perl as a set of operational tasks, suited to our peculiar structure, needs and purposes. Most of the underlying code has been written by Michael Jensen, who while not a professional programmer, uses code to solve publishing problems. The code wouldn't pass Perl/Javascript 101 for elegance, but it gets the job done. We are willing, even interested, in making parts of this suite (for example, the keyphrase-weighter and extractor) available in open source, so it could be improved by real programmers with a purpose, and thus made more efficient and worthy of reuse by others. See Michael's page to contact him about this option, if you're interested.

Can I link to a "search builder" from my own site?

Please do. We hope to see this tool being widely used to assist in better use of our publications, and the content within them. Simply copy the URL listed in your browser, for any chapter's "search builder" interface.