Dealing with data deluge

More information was generated between 1999 and 2003 than otherwise was created since the beginning of mankind, declares IDC of Framingham, Mass., a high-tech research company.It's anyone's guess how much this equation has changed in the three years since then, but the trend is clear'we're flooded with new information daily. Nearly 4 trillion documents are created yearly in North America alone, and this figure is rising steadily.The problem for most agencies lies not only in the proliferation of new information per se, but in how successfully it can be managed.Consequently, despite the growth of new technologies, such as vastly improved computerized storage capability, the use of the Web as a business tool, the rise of e-mail and dozens of other information hardware and software advancements, information management remains a paramount issue in virtually every enterprise.In many offices, you'll see information workers searching through paper stored in overstuffed file cabinets, stacks of microfiche, electronic data in file servers, e-mail and pages of text in various electronic formats such as Adobe Portable Document Format. Despite the use of old and new storage technologies, the result of many information searches is all too often the same'tedious, time-consuming and fruitless.'If you can't find it, you can't use it' is a truism that by now should be etched firmly in the minds of IT managers, who may or may not realize how critical sound information management policies are to them and their agencies.That said, a host of hardware and software tools exist that, when judiciously combined, can greatly improve your chances of finding, saving and reusing information that may be critical to your business goals.On the hardware side, digital scanners can render material into graphic or text files for storage in enterprisewide archives or in various document management systems for further manipulation or enhancement.On the software side, a wide range of document management systems can work hand in hand with scanners to manage the creation, storage, safety and reuse of information critical to the enterprise.Together, these tools can revolutionize the ways your agency manages the tide of information coming its way.Scanners can be attached to workstations or personal computers to digitally gather information in the form of text or graphics that can be stored or otherwise used as you see fit. There are several types of scanners available, from relatively low-speed home versions to high-end, ultrafast product models, handheld models or those designed principally to handle photos or transparencies.For the accompanying product list on Page 22, I chose midrange flatbed scanners with components designed for relatively steady workgroup or departmental service. These scanners boast an average speed of 40 to 60 pages per minute. With several exceptions, the systems selected provide automated document feeder capability, which enables documents of similar size to be automatically fed into the scanner. The ability to scan in duplex mode (two sides of the page simultaneously) will prove advantageous for most workstations or departments.Scanning resolution of between 100 and 600 dots per inch is typical, with a resolution of 300 dpi being adequate for most jobs except for the finest of detail, where 600-dpi resolution is called for. User-selectable resolution is desirable.The ability to scan in color images is valuable, although for extremely high-speed tasks, black-and-white renderings are fine. In most cases, A4 document size (8 by 14 inches) is fine; for ledger-sized documents, up to A3 (11.7 by 17 inches) can be managed by many of the units listed.The ability to scan even larger images is available in devices from many of the vendors listed in this guide, but at a price premium. For illustrative purposes, the PS7000 from Konica Minolta Business Systems of Ramsey, N.J., which can scan in images as large as 17 inches by 23 3/8 inches, is included in the list.All the scanners in this guide come with many functions besides ADF, color scanning and others. If documents are to be saved as text files (thus enabling text search and editing), optical character recognition software should be bundled into the system, as it is with virtually all the scanners listed.[IMGCAP(2)]Another very popular format for document storage is Adobe Portable Document Format. PDF is like a picture of the original document, because all graphics and text remain as they were in the original. Since free software is widely available to read it (Adobe Reader), PDF has become a de facto storage standard, whereby the original document and all its notes and markings remain intact.Most of today's workgroup/departmental scanners come with both SCSI and USB interfaces. Since most scanners are connected directly to PCs or workstations, you'll like want SCSI or USB connectivity. For standalone network connectivity, the scanner will have to come bundled with a network interface card.TWAIN and ISIS are the main interface standards for scanners and digital copiers. TWAIN drivers let you scan an image directly into an application such as Adobe Photoshop, where you can work with the image. These drivers are generally bundled with other software coming with the scanner. ISIS is a similar standard.Two big factors to consider when choosing a scanner are quality and speed. If you want to scan documents so the full text is searchable, you'll need OCR, which requires a higher-quality, more-expensive scan engine than a device for simply storing documents as images.As for speed, you can buy a fairly fast scanner at a relatively low price, but make sure its components are hefty enough to handle a steady volume of pages at maximum speeds. For example, a small, 25-ppm flatbed is theoretically capable of scanning 12,000 pages per day but it is probably rated for a daily duty cycle of 1,000 pages. It will break down under heavy daily traffic loads, where a more expensive machine with a larger duty cycle won't.As for cost vs. performance, if you need to scan millions of pages, you'll want a high-end scanner that can manage up to 150 ppm. This machine, however, may cost $80,000 or more. It might be better to buy multiple scanners instead of one high-priced one. Three 40-ppm scanners priced under $5,000 cost far less than one 120-ppm device. Plus, three people can work on scanning simultaneously and you'll have two running if one is down for service.The document imaging/management products in this guide were selected for their general ability to work with scanners and to perform the functions most buyers have in mind when seeking information management solutions. Most of them are client-server solutions; that is, they are locally hosted (run on your own computers) and designed to store data on your own hard drives or on a network server in your office.While there are hundreds of document management software packages out there, it's important to bear a couple of points in mind in sifting through your options.First, keep in mind the distinction between document imaging and document management software. The goal of most document imaging solutions is to efficiently convert paper records into electronic files. Document management software is designed to help manage these electronic files.That said, the two types of software are similar, and the distinction between the two types is blurred in many products. For example, both types of software require security, tracking and indexing features. And many enterprise document management packages bundle strong document imaging features into their modules.Knowing that it's going to cost your agency thousands of dollars annually to embark opon a document imaging/management solution, what are the reasons for doing it immediately? Here's a rundown:. Despite high startup costs, one of the biggest hidden costs of running a paper-intensive outfit is the time required to manage those paper files. Inc., a Web-based reseller of document management hardware and software, estimates that a $20-per-hour employee retrieving 10 files per day from a storage room may use up to 216 hours per year at a cost of $4,320. This is the cost of information retrieval for one employee only'imagine the aggregate cost of hundreds of employees performing the same tasks over years of service.. Document management solutions provide the best way to ensure compliance with the wide variety of rules and legislation that affect most business and government agencies today. Instead of ad hoc compliance measures to meet the requirements of, say, the Patriot Act, these steps may be automatically built into the software and customized for use by a particular group within the agency. Consistency in such measures as enforcement of naming conventions ensures that various approval processes may also be complied with in ways difficult to achieve in the typical paper-driven environment.. Electronic document management provides several layers of security not readily available in traditional physical document storage/retrieval systems. These include multiple levels of password-protected access, encryption of document contents and electronic audit trails showing who has accessed or updated documents.. Electronic document management systems protect paper records by creating electronic copies that can be readily backed up. This includes off-site data backup that ensures that a fire, flood or other disaster won't cripple your agency.. A good document management system enables employees to access records from wherever they are. Multilayered access enables them to see and make changes to the documents they are authorized to see.