The AllTheWeb.com site had previously offered multimedia searching but searches had to be retyped for each kind of search. Now the site offers integrated multimedia, MP3, and FTP results. For example, a search for "Wimbledon" will display the normal pages found (in either Full Description or Title-only format) and will provide a box with the results for pictures and videos of the event.

Users are now offered automatic search tips, such as suggestions on limiting by language or using phrases with quotes. Claiming to be a search engine first, AllTheWeb.com uses a set of artificial intelligence algorithms to analyze user behavior in real time and provides tips to help users modify and narrow their searches. In addition, on the results page, a "Narrow Your Search" feature suggests specific terms that can be simply clicked to activate a search. In the Wimbledon search, suggestions include wimbledon tennis, wimbledon theatre, and the wimbledon fc (football club).

Tom Wilde, vice president of marketing for FAST, said that the biggest challenge for the company today is user education. A user types a one- or two-word query and hopes to get good results. The suggestions provided to users are a way to involve them at a deeper level and improve their search experience. He said that AllTheWeb.com uses "site collapsing" of results, which organizes results from the same Web domains together and then offers a way to click on "more hits from…." This makes the display of results less cluttered and helps users find what they are looking for faster.

The relaunch also provides a new version of FAST's FirstPage technology, which enhances the relevance of results. This leverages multiple results-ranking methodologies, including advanced link analysis. FirstPage is actually a premium index of about 3 million pages. Wilde noted that this is most effective for identifying company Web sites and proper names.

Wilde also noted that the upgrades to the search provide ongoing incremental improvements in reducing levels of spam, duplicate links, and offensive content. The latter, he admits, is a difficult problem, but one that the company has been working on. The algorithm is more complicated than simply blocking keywords. It works from a dictionary of identified terms and does proximity and frequency matching for ambiguous words.

For power users and complicated searches, it's worth choosing the Advanced Search option, which lets users refine a query and provides very specific word filters, domain filters, and results restrictions. The engine supports 46 languages and this can be specified from both the home page simple search and the advanced search page.

The company claims that its FAST Web search engine offers the largest, freshest, and fastest catalog on the Web, spidering over 1.6 billion Web pages to create a high-quality catalog of 625 million full-text indexed pages. The catalog is refreshed every 2 weeks, while the company claims its competitors refresh every 4 to 5 weeks. Currently its 150 million FTP and MP3 files are refreshed twice a day, while the multimedia data—pictures, video, and sound—are updated every 2 weeks.

Wilde said the company plans to issue quarterly releases with new features and enhancements to its search site. Plans for the future include incorporating thesaurus look-up capabilities and a spell-checker that would assign the correct search terms.

In June, FAST announced its paid inclusion program, called FAST PartnerSite. It's now in beta and will be rolled out in September. Normally, the search crawler revisits pages every 2 weeks. For $99 a month, submitted pages will be included in the FAST index and will be revisited every 24 hours. Custom pricing is available for larger sites with many pages. Webmasters of the PartnerSite program will also be provided with "Site Search" functionality through a search bar that delivers searching on their sites. Note that this does not affect relevance or positioning of the results—it ensures inclusion, not placement.

Wilde said the company is pushing hard to improve and enhance the interactions with users. Improving relevance is a continuous emphasis. Currently the FAST engine doesn't have a natural language algorithm, but Wilde noted that retrieval is actually quite good for such queries. One of its partners is Albert, which now has a demo site that uses the FAST index and layers the Albert natural language algorithm (http://www.albert-inc.com/demo/php). Wilde said the company may consider adding some natural language capability to its search engine in the future.