FirstGov moves to cluster technology to expand search engine capabilities

Users of will soon have access to significantly more online government data, thanks in part to an expanded search function, General Services Administration officials said.Mary Joy Pizzella, associate administrator of GSA's Office of Citizen Services and Communications, said a new contract the agency awarded to Vivisimo Inc. of Pittsburgh to expand and compartmentalize FirstGov's search and information portal will significantly improve access to federal information.This project 'is going to expand the amount of information [available on FirstGov] tremendously,' Pizzella said.Under the $1.8 million annual contract, Vivisimo and its subcontractor Microsoft Corp. will add numerous resources to the existing FirstGov search function and let users search for data across all levels of government'including state, local and tribal authorities.GSA awarded the contract last month as part of an $18 million, five-year blanket-purchase agreement for Web search services. Three companies including Vivisimo were selected to participate in the BPA after the agency issued a request for quotes in the spring, Pizzella said. The other companies are Gigablast Inc. of Albuquerque, N.M., and Fast Search and Transfer, a company based in Norway with its U.S. headquarters in Needham, Mass. FAST currently operates the site's search function.GSA structured the BPA so each of the three companies bid to provide the search services each year, meaning that while Vivisimo won this leg of the competition, the companies can revamp their products and make another run next year, Pizzella said.The new search function will go live next spring, followed by video and news search in the fall, GSA officials said.FirstGov already serves as the government's chief information portal on the Web. The site acts as a gateway to other sites so users can troll about 8 million federal Web pages for basic information, including jobs, regulations, assistance and disaster recovery programs.The new system will expand that database more than threefold, officials said, and will add features such as news and images that are unavailable on the current site.'This is going to allow users of FirstGov [to search] not just site-by-site, but across the whole [government] index,' Pizzella said.While GSA and Vivisimo hope the project results in a better user experience, search expert Chris Sherman said most FirstGov visitors probably won't notice a difference.'I think they're going to improve [FirstGov], but I don't think that we're going to see any significant change in the quality of search results,' Sherman said.Sherman, president of Searchwise, a search engine analysis company in Boulder, Colo., said that while Vivisimo will make incremental improvements, the overarching factor in awarding them the contract was price. The current search engine is working quite well, Sherman said, so the decision came down to money.'Vivisimo came down with such a compelling offer that [GSA] decided to go for it,' Sherman said. 'I'd be surprised if they were dissatisfied with FAST.'The search function will rely on software developer Vivisimo's Clusty.com search engine and incorporate results and features from Microsoft's MSN search tool, GSA said. As an example, GSA officials said that if a user searched for 'nursing jobs,' results would come from both the FirstGov.gov site and the database at .Search results will return both in a list format and grouped in clusters by subject, letting users choose a random sampling of returns or a more organized collection.Raul Valdes-Perez, Vivisimo CEO and co-founder, said the clustering technology will make it easier for FirstGov users to find what they're looking for.'When it is all said and done, you get a lot of stuff from everywhere,' he said. 'We can organize it by theme. ... All of that put together will represent quite an advance for FirstGov.'Vivisimo will hire an Internet librarian to collect information and identify resources for the search engine, presenting volumes of data to the user, he said.The engine will also use MSN's search tool, bringing news and other data as well.The partnership with Microsoft gives the government portal a wider and more thorough view of search functions, Microsoft executives said.Federal enterprise search engines generally do not have a World Wide Web outlook, so adding MSN's engine brings a much broader vision, said Aravind Sampathkumaran, group product manager at MSN's Information Services division.'The technology [FirstGov] uses today doesn't get the benefit of a Web-based search experience,' he said.MSN 'pretty much covers the Web,' he said, and does a better job of retrieving federal documents than the government.GSA selected FAST's Enterprise Search Platform as the content manager for FirstGov.Under the contract, also signed as part of the BPA, FAST ESP will provide FirstGov with a method for storing and making available documents published on the site.