Enterprise search vendors work continuously on improving the technologies and features in their products to scale to the exponential growth in content and demands for better answers. One vendor estimated that the volume of data in a large organization doubles every year. A company's ability to compete effectively hinges on the quality and efficiency of its information retrieval solutions. The Norway-based company Fast Search and Transfer (FAST; http://www.fastsearch.com), which has been one of the leading developers of enterprise search technologies, has just launched FAST ESP 5, the next generation of its Enterprise Search Platform. FAST ESP 5 provides organizations and their external and internal users with a secure, unified point of access to all structured, unstructured, and rich media data. In addition to improvements in scalability, usability, and performance, FAST ESP 5 uses a new technology called Contextual Insight to return answers to questions, not just a list of references. The company and some industry observers are hailing this release as "changing the game."
"As the leader in the enterprise search industry we transformed the landscape 2 years ago when we brought our first version of FAST ESP to the market. Our platform's unified view of enterprise information, its security, scalability, and usability, set a new standard for intelligent search," explained Ali I. Riaz, president of FAST. "Today's launch of FAST ESP 5.0, with Contextual Insight and its extreme scalability and superior capability to build differentiating solutions, will profoundly change the way organizations do business."
"Contextual Insight is a major leap forward, and will truly change the way users can search for information," commented Carel de Bos, CIO of Reed Business, a division of Reed Elsevier.
So just what is this amazing Contextual Insight? It consists of a fairly complex group of technological improvements, but basically FAST has added linguistic and statistical analytics to improve precision.
Contextual Insight introduces a semantic index that recognizes and retains the inherent structure in a document and scopes the search to the structural element level. Conventionally, the scope is a document, database record, or Web page, but these are too ambiguously large. With Contextual Insight, the scope can be a sentence or paragraph or, in general, any XML or structural element. Contextual Insight annotates the scope with entity metadata by identifying all the entities (names, companies, locations, etc.) it can find to provide the answers. FAST ESP 5 supports about 40 entities and the list is growing. There is even a standard, open interface for defining your own. It also added other context-aware features, such as geospatial location, patterns, profiles, etc.
FAST ESP also provides contextual navigation capabilities to let users explore the content. You can see this at work in sites like Scirus.com (from Elsevier), ThomasNet, and CareerBuilder. Here's how FAST describes the three navigation types:
- Taxonomic navigation uses concepts and categories from a taxonomy as navigators. Documents are auto-categorized under these categories.
- Faceted browsing exposes database table fields as navigators. The navigators are the possible values as defined by the results of the previous search.
- Entity discovery extracts textual entities from the results of the previous search. Unlike taxonomies and facets, entity extraction draws its navigators directly from the results and is contextually aware.
Susan Feldman, research vice president for content technologies at IDC, has
said for some time that FAST offers an advanced integrated platform that combines tools for finding both unstructured content and structured data. When FAST launched ESP in January 2004 she called it "a leap into the future" with its unified view of enterprise information.
Feldman is even more impressed with ESP 5. "ESP 5 is an example of a new approach to information access-an architecture that combines both data and content in the same extended index. It no longer requires that the search interface send separate queries to data and content indices and then merge the results. As a result, the system is faster and more scalable. And, it can calculate relationships between the entities it extracts on the fly, making it more flexible."
"The game has changed," she told me. Databases are not usually good for ad hoc questions, she explained. Users have to predict ahead of time what they want to know and reports must be predefined. FAST has now extended the search index to create a hybrid file structure that allows average business users to create queries and reports on the fly.
According to Feldman, Endeca is the only other company that can search across data and content. But FAST has added the ability to extract entities and do on-the-fly calculations of relationships within the search engine.
"This development comes from the enterprise search side but it is actually a threat to the Business Intelligence market-and this could shake up the market. Search technologies have become vital to enterprise computing because they provide language understanding. As applications become more interactive, search will be embedded in everything," she predicted.
Factiva plans to use the entity identification capabilities in ESP 5 for its recently launched Search 2.0 (see the NewsBreak at http://newsbreaks.infotoday.com/nbreader.asp?ArticleID=16009). According to a Factiva representative, Search 2.0 was designed with ESP 5 in mind. FAST and Factiva collaborated on ESP 5 capabilities and Search 2.0 is scheduled to leverage ESP 5 in the first half of 2006.
"At Factiva we are constantly advancing our offerings and our capabilities to meet the needs of the rapidly changing marketplace. FAST shares this vision, and its solutions have greatly helped us revolutionize the way we deliver business news and information to our customers," said Clare Hart, president and CEO of Factiva. "Our relationship is a perfect example of how innovative technology can contribute to creating competitive advantage."
FAST just reported record revenues of $31.1 million in Q4, yielding growth of 52 percent and record revenues of $103 million for 2005. The company also said that it signed a record number of new customers in Q4 2005, surpassing Q4 2004 deals by more than 250 percent.
FAST boasts a number of well-known clients in our industry space, including LexisNexis, Factiva, Reed Elsevier, Reuters, and more. Another successful project reportedly delivered on time in 2005 and with high quality was one for NewsBank. FAST ESP is powering its Web site, which features more than 140 million articles from more than 2,000 U.S. and international sources. Also, FAST and LexisNexis just announced last week that LexisNexis Taxonomy Solutions would be offered through FAST ESP.
FAST's vision has been to transform enterprise search from a powerful, yet narrowly perceived function into an enterprisewide strategic and tactical capability of pivotal importance. With ESP 5, this looks quite possible.