4 challenges to realistic records management

Data analysis has become a twisting Mobius strip, looping back not only to influence not only how we look at data but also how we manage data itself. That makes deriving value from content anything but an easy process.

Despite the rapidly advancing processing and algorithmic tools available today, organizations are having a difficult time reaping the insights that these technologies are supposed to generate. This is largely because of to the mismanagement of data; data that hasn’t been managed and cleaned is of little use for analytics, no matter how slick the user interface is. Unstructured data, in particular, is resistant to uniform management. Its multiformat nature and ongoing, rapid generation spawn a wildly diverse and rapidly evolving ecosystem of content. Nevertheless, structured data is critical to daily business productivity and is needed for meeting legal and regulatory requirements. It also has immense potential for business insight.

At the heart of this struggle are the records and information management (RIM) professionals, who experienced a rapid metamorphosis of their roles as data volumes expanded and paper dwindled. Legal changes a decade ago prompted a whack-a-mole approach to information management, segregating data into separate systems, depending on need. The paradoxical result of these data management silos is an environment that is even more expensive and difficult to manage. With guidelines and objectives now cemented via the Managing Government Records Directive (M-12-18), agencies are faced with the daunting task of implementing seemingly “simple” requirements that are actually quite complicated, given how disparate data has become.

We’ve entered a new era of RIM, with big data at the forefront. Paper is out; analytics are in. RIM professionals are eager -- rather than apprehensive -- to use partial automation to tackle the volumes of data that they could never have classified by manual means alone. But there is still a layer of frustration. Top organizational leadership often champions a holistic and singular management strategy for data, but those in the trenches know that the nuts and bolts of implementation can become a nightmare, drawing out original costs and timelines.

Some common major challenges were widely debated at the recent 2015 ARMA Conference for information governance professionals. One hot topic was the role of RIM in organizationwide governance -- that is, whether and where RIM sits at the table, as either a standard participant or host. I suspect that the end result will depend on the dynamics of individual organizations, the clout that RIM has established within the organization, how much RIM is willing to take on and how much the organization will allow RIM teams to control such an endeavor.

At the conference, other challenges facing real-world RIM teams surfaced as well. The debate over deletion was still hot, but data mining and low storage costs have given new vigor to the “hoarding” school of thought.  That was only one challenge discussed, however; several common themes were debated, all with big implications for RIM. 

Challenge #1: Deciding what to delete

The rise of analytics capabilities has created an ideological impasse between organizational units. More traditional, risk-averse units such as RIM and legal teams strive to eliminate outdated content as soon as it is legally permissible to do so. More proactive factions -- such as marketing and management -- want to keep as much data as possible to leverage in analytics: the “more is better” approach.

It’s a paradox. Analytics conventions seek to amass data, while traditional data governance seeks to systematically eliminate unneeded content. Regardless of where an organization falls on the “risk tolerance” spectrum, the first step is to decide what (and when) to delete. But even in order to eliminate some data, the organization essentially needs to “touch” every item of content to decide whether to eliminate or retain it.

The most elegant solution would be a singular environment where all unstructured content is handled centrally: where enterprisewide policies for retention can be programmed and executed consistently and where no duplicate copies linger in the shadows.

Challenge #2: Designating consistent access privileges

Having data scattered across departments and silos creates a barrier to implementing consistent access privileges for users. Within a single application or silo, it’s typically straightforward to assign correct access rights. That falls apart, however, when multiple silos are added. An individual who may have the right level of access in one system can be completely blocked in a related platform -- or, conversely,  may be given far too much access. The coordination of permissions across platforms often takes manual updating, which quickly becomes unfeasible due to lag times and high levels of human error. To complicate matters, permissions often are based on variables that may change over time, such as timestamps, and the same document may have inconsistent policies across different applications. In a siloed environment, access privileges have little meaning.

Challenge #3: Maintaining complete audit trails

Many compliance and legal uses for data require a comprehensive log of changes, edits and ownership for a given piece of data -- without gaps in ownership. These audit trails are critical for defensibility, but they are nearly impossible to maintain when data copies reside in multiple silos. The audit trail for a particular item is only relevant for the actions taken in the exact same platform in which audit trails are generated. A standard Word document may be deemed relevant as a record and placed into an isolated records tool – such as an enterprise content management system – but it now has multiple copies and identities.  Inevitably, the audit trail provided on each copy will be different. To definitively know the full history of an item of data, there must be a single point of contact: a single “master” environment into which all relevant items are stored as single copies, even if duplicates exist elsewhere.

Challenge #4: Scaling up classification

Today’s volume of data simply cannot  be effectively classified manually. The ever-growing volume of information, changing policies and legal precedents that deem nearly all data discoverable have completely redefined what constitutes a record. Even if a company sets defensible policies for the expedited disposition of “junk” content, the problem remains that every single piece of data must be assessed in some way in order to determine its status.

And since every piece of data in the organization must be “touched” at some point, a blend of automatic and manual classification is currently the only feasible approach. The exact configuration and division of roles between man and machine will depend on a specific organization’s needs. For now, it is generally agreed that records professionals are best left to do what they do best: assigning policies for complicated or ambiguous items. Autoclassification can then be left to filter out the easily identifiable items such as those with specific URLs, items with certain metadata or items created by predefined key individuals. However, in order for this to happen, there needs to be a single classification engine through which all data passes. Separate systems with unique classification capabilities cannot create a consistent result, causing irreconcilable holes in defensibility.