The long-awaited unification of the Dialog and DataStar services was not a trivial task to achieve. It had been attempted several times in the past but not delivered due to the challenges of redesign and reload for the enormous amount of content involved—said to be more than 1.5 billion unique records. With the benefit of ProQuest technology resources, it’s really happening.

Dialog staff spent the first year after the acquisition talking to customers and getting feedback. This is what they heard their users wanted:

• Need to maintain depth and breadth of content
• Make precision search easier, both for trained searchers and for end users
• Add presentation, collaboration, and analysis tools
• Keep pay-as-you go pricing
• Leverage ProQuest content and technology

As Lynn Christie, Dialog VP of product management, says, “Our prime directive was to improve ease of use without sacrificing search precision.” It’s really about keeping two types of users happy with one system—highly skilled information professionals who like precision and control and end users who needed an easier search interface.

The developers spent a lot of time looking at search logs and observed how users searched and came up with an “Innovation workflow cycle”—it’s designed to support innovation across 3 user areas:

• Discover – R&D kind of research (STM content)
• Validate – legal professionals looking to validate uniqueness and patentability (intellectual property)
• Market – business intelligence and marketing professionals track competitors and industries and bring new products to market

So, ProQuest Dialog is being released in three streams of product development, in the order above. The first installment of Discover covers pharmaceutical and biomedical content. Databases include BIOSIS Previews, Cab Abstracts, Derwent Drug File, Embase, MEDLINE, Gale Group PROMT, etc. The rest of STM content in the Discover phase will come out later in 2010. Validate and Market content will follow in the first half of 2011.

The initial summer release has an intuitive interface with powerful search refinement capabilities (limiters), an advanced mode for query building, and new capabilities for managing search results. The summer release features have been modeled on the new ProQuest platform that is under development.

• Intuitive interface that offers a radically improved user experience
• Advanced search features such as Rank are built right into the interface (not separate commands)
• Enhanced retrieval (especially to support end users) through suggested terms from a drop-down list, recognition of synonyms, word stems (supports left-handed truncation as well), and misspellings (did you mean?)
• Easy access to thesauri to improve cross-database searching
• Normalized fields to facilitate post processing

The summer release is primarily an end-user tool but trained searchers may want to use it as an easy, yet powerful research tool and may also be interested in the collaboration tools. Fairly powerful Boolean-based searching is available in the current Advanced Search option. But true command-line searching (Classic Dialog) will come in a later release. The summer release only supports subscription models

An information center manager at a large pharmaceutical firm wrote, “Very user friendly! Dialog is approaching the new product from the end-user POV, not the Information Specialist or Librarian. Elegant and it works.” Customers at 9 large companies around the world conducted beta tests in May

Later releases will bring additional engineering, technical, and scientific resources, offer a command-based search option, and provide direct access to full text and other content from ProQuest. Transactional pay-as-you-go access and usage tracking will be offered

Amelia Kassel, a long-time Dialog user and expert searcher, was on an expert panel 18 months ago, convened to pick their collective brains for what they wanted in the new system. One of her biggest concerns was availability of transactional pricing. “And, we got everything we asked for.” She said, “It’s already a very powerful system—I could use it right now. Even expert searchers want services that are easy to use.” She has no complaints at this time, since she hasn’t had a chance to test it—except that she’d really like to have demo accounts available for the online searching course she teaches in the library school at San Jose State. She plans to beg and plead for spring 2011 availability

Another expert searcher who was on the panel, Marydee Ojala, editor of ONLINE, says, “With this new platform, Dialog is trying to move professional search into the web world, where Google has trained us to expect a single search box, while not losing the advanced search capabilities information professionals rely on to conduct effective Dialog/DataStar research. It has not been an easy task. Dialog has been diligent in testing its platform on real searchers, listening to their reactions, and altering the platform as needed.”

I had a pre-launch preview of the service and then got a production release password to test it. While I’m not a pharmaceutical searcher, I did enjoy trying some searches and playing with the interface. It was easy to just click and go—very intuitive and it offered fast response times. One of the limiters to narrow results is a date range option displayed as a bar chart. Hovering over one of the years displays the number of records available in that time frame. Drag the slider to change the time frame. Cool. There’s also a Find Similar feature that lets you paste in text from a document to find similar items. You can search within your search results, apply various limiters, and easily see what you’ve done and remove anything you don’t want. Another nice feature—while hovering over the Preview button for the article in the results set, a box with the summary will display. When viewing an article, a box in the right hand pane suggests “other items you might like,” based on metadata.

The summer release includes an export citation link directly in the record to RefWorks or other bibliographic managers like ProCite, EndNote, and Reference Manager. You can also cite an article in a number of formats, save as a file, email, print, add tags, and save it to a personalized workspace called “My Research” where you can also manage your alerts and RSS feeds.

There are a lot of new features. For the details, check out the FAQ. There’s also a Fast Start Guide. Other documents include a Quick Reference Card and Search Tips that can be downloaded.

ProQuest’s Platform Plans

ProQuest has just started previewing its major new platform release that includes a very extensive collection including all content from ProQuest and CSA plus some from Chadwyck Healy. Library administrators in the ProQuest customer community are being invited to try out the new platform, become familiar with its capabilities, and prepare for full roll-out and migration to the new platform for their student and faculty users in the coming months. I did not get a look at this yet. Watch for our ongoing coverage of the roll outs.

Here’s the plan, according to Lynda James-Gilboe, senior VP, marketing and customer care: “ProQuest is taking a phased approach to migrating customers to the new platform, allowing customers ample time to preview it before they migrate. We alerted them to the migration schedule in July and [during] the week of August 23 the first wave of the preview period is opening for customers. We’ll continue rolling the preview out further through late summer. Doing this in waves allows us to provide customers with effective help and answer questions without delays.”

The New Generation

ProQuest CIO Bipin Patel says it was a “huge technical challenge” to engineer the “refresh” that had to occur to create the new underlying technology infrastructure out of multiple legacy platforms. The company needed to start fresh from the ground up. It built a new XML datastore that Oracle says is the largest XML database. ProQuest tripled its search capacity, which is powered by Microsoft’s FAST search engine. It simplified the architecture, built common services, and worked to normalize and map files. Patel says the company is loading 10-12 million documents per day onto the new platform. The new infrastructure is stable, flexible, and agile, so that significant enhancements can be introduced every 2 or 3 months. The company has logically scaled back work on the legacy platforms and has been “sunsetting” some old products, such as DialogSelect Open Access.

Web search engines have certainly raised searchers’ expectations. ProQuest and most of the other information companies have been working feverishly to bring their systems up to current technology standards with complete platform re-dos, new architectures, intuitive user interfaces, agile development environments, and more. Any companies that are not doing this are likely to be left in the dust. Customers should welcome this new generation of products with open arms.