Rethinking the Structured–Unstructured Gradient: Why Records Managers Must Stop Treating Data as “Not Our Problem”

How records managers lost half the information landscape and how we can reclaim it

Sometimes a single idea catches you off guard and refuses to let go. You read it once, nod at the obvious truth, then find yourself thinking about it in the middle of a meeting or on a quiet morning walk when your mind drifts toward bigger questions.

Karl Melrose did that to me with his October 2025 Meta-IRM piece, “The continuum we should be thinking about in records and information management.” It was not flashy or loud. It was the kind of insight that feels so intuitive once stated that you wonder how we failed to articulate it all along.

His message, in essence, was this:

The real terrain of records and information management is the gradient between more structured and less structured information.

Not a boundary. Not two camps. Not an either-or. A gradient.

A landscape.

Once you start seeing the world that way, it is very hard to go back to the old maps.

This article is my attempt to walk through that landscape with you. Think of it as a guided ramble along the structured–unstructured gradient, with stops along the way to bring in the theory, the standards, the metaphors, and the practical implications that Karl’s piece opened up.

Along the way we will encounter some uncomfortable realizations about how the profession has behaved, but also some clear opportunities for rebuilding trust and relevance.

So let’s lace up and step onto the trail.

---

Where it starts: structure, noise, and the places we avoid

Karl opens with an observation that anyone who has ever tried to find meaning in a mess of files or a poorly designed database knows in their bones.

Information without structure is noise.

We know this. Every time we confront an untagged file share, or a wandering email thread with no subject line, or a system that lets people type anything into any field, we feel the drag of disorder. Structure is what turns clutter into knowledge. It is the scaffolding that lets information stand up straight.

Which is why it is striking that the profession has been so selective about where we honor structure.

Give us an EDRMS and we will happily apply structure all day long. We will create metadata schemas and approval workflows. We will design classification hierarchies to support retrieval and disposition. We will insist that users follow naming conventions. We will declare victory when a document is “captured.”

But show us a CRM full of complaints, or an ERP full of financial transactions, or a case-management tool full of decisions, and too many records programs still respond with something like, “That belongs to IT.”

The irony is that these structured environments have stronger built-in control, context, lineage, and validation than any document repository. They are often the real places where business happens and where evidence is created.

So why do we pretend they sit outside our field of view?

Karl gestures toward the answer: habit, history, culture. The places we avoid are rarely the places that lack structure. They are the places where we have decided structure is not our job.

And that decision has consequences.

---

The theory, the standards, and the global practice all say the same thing

Once you step into the research, Karl’s insight starts to feel almost overdetermined. Everywhere you look, someone has been telling us this for years.

The standards have been waving red flags for two decades

ISO 15489 told us in 2001 and again in 2016 that records are defined by function and evidential value, not by format. It told us that systems matter, context matters, and structure matters regardless of medium.

The Records Continuum Model, developed in Australia, tore down the idea that records move from creation to archive in neat, staged lines. Instead, it insisted that recordkeeping is woven through systems, processes, and actions as they occur. It is embedded, not appended.

And ISO/TR 8344, published in 2024, puts the matter bluntly: structured data environments produce records. They are among the most important recordkeeping spaces in contemporary organizations. If you ignore them, you compromise integrity, authenticity, and accountability.

In other words, the standards have been holding a mirror up to us for years. We just have not always liked the reflection.

The literature is even more direct

ARMA, AIIM, NIST, national archives authorities, and academic scholars all converge on variations of the same idea:

• Everything is information.

• All information sits somewhere along a structured–less structured gradient.

• All of it can become a record depending on its function.

• Governance breaks down when we treat only one region of the gradient as legitimate.

Practitioners like Daragh O Brien go further, arguing that the entire structured–unstructured distinction is now a false dichotomy. AI can impose structure on text. Humans impose meaning on data. Nothing lives entirely at one extreme.

Once again, the gradient appears.

Governments around the world have already accepted this

Across jurisdictions, public-sector practice reinforces the point:

• South Korea preserves structured datasets as official records.

• Estonia treats microservice transactions as authoritative documentary traces.

• The European Union’s GDPR forces unified treatment of personal data across formats.

• The United States federal records directives explicitly include all digital information, including databases.

The world is not debating whether structured systems produce records. The only debate is whether records managers will acknowledge it.

---

Seeing the gradient clearly: not two sides, but movement

The moment you stop thinking in terms of “structured” and “unstructured” as categories, the landscape comes alive.

Documents have embedded metadata, version histories, layout structure, and often hidden XML layers.
Databases have comment fields, attachments, annotations, and non-normalized text.

Almost nothing sits in a pure state. Everything exists as a blend.

Even the way information moves across the gradient is fascinating.

A messy Word document becomes more structured when:

• you extract entities

• you tag it with metadata

• you apply a workflow

• you link it to related content

A structured table becomes more narrative when:

• you generate a report

• you create a dashboard

• you explain its business rules

• you migrate it into a preservation package

The gradient is alive. Records move along it. Interpretation moves along it. Technology moves along it.

And so should we.

---

How we inherited imaginary boundaries, and why they are failing now

The divide did not come from theory. It came from history.

For decades, records management grew up in the shadows of paper, filing cabinets, classification schemes, and archives. We knew how to work with documents. We had decades of institutional memory in that space. We had a language for it.

Data management came from computing. From structured systems. From relational models, SQL, schemas, and storage architectures. Different vocabulary. Different value system. Different professional home.

So two worlds evolved side by side.

For many years the separation held. It even made sense in the early digital decades, when business systems were limited and document stores were dominant.

But that world is gone.

Today, the core evidence of almost every business action lives not in a file but in a system.

A refund is approved inside a POS application.
A permit is issued inside a workflow tool.
A compliance incident is logged inside a case-management platform.
A contract is born, maintained, amended, and completed inside a procurement system, not in a neat folder structure.

If we limit ourselves to the documentary region of the gradient, we are looking at the shadows of business activity rather than the activity itself.

That is not records management. That is recordkeeping tourism.

---

Stepping onto the gradient: what it looks like in real practice

Accepting the gradient means changing our behavior.

First, we start by seeing systems as record environments

A CRM is not just a customer database. It is a recordkeeping environment for complaints, approvals, escalations, and resolutions.

An HRIS is not just a personnel platform. It is evidence of employment decisions that can affect careers, litigation, and rights.

An ERP is not just a financial backend. It is an auditable chain of obligations, payments, and responsibilities.

If it holds evidence, it holds records. The gradient makes that a starting assumption, not an exception.

Second, we apply retention and disposition to structured data

This is one of the hardest shifts. Retention scheduling for databases requires:

• identifying business entities and fields

• mapping them to record categories

• designing purge or archive functions

• documenting schema changes

• exporting records with metadata, not just data

ISO/TR 8344 is crystal clear that these tasks belong inside the governance ecosystem.

Third, we design systems with recordkeeping in mind

This means:

• including recordkeeping requirements in procurement

• collaborating early with architects and engineers

• requiring audit trails, integrity checks, and provenance metadata

• ensuring that data structures remain interpretable over time

It is the “records by design” principle made real.

Fourth, we use the gradient rather than fighting it

Documents become more manageable when we add structure. Data becomes more meaningful when we add narrative.

Governance improves in both directions.

---

The accountability moment: why this matters now

Karl does something important at the end of his piece. He reframes the question of accountability.

People often tell us that records management is about accountability. Karl suggests that the situation we are in right now may represent our organizations holding us accountable for failing to show up across the full informational landscape.

This is not a comfortable idea. But it rings true.

If we decline to engage with structured environments, we are partly responsible for:

• compliance failures

• privacy exposure

• poor digital preservation

• incomplete audit trails

• weak appraisal decisions

• unmanageable legacy systems

• data that outlives its legitimate purpose

• records that lose meaning because their context disappeared

Trust erodes quietly, then suddenly.

The good news is that trust can be rebuilt when the profession shows up fully and confidently across the gradient.

---

A final note, and an invitation

Karl’s insight is important. So is the timing. The shift toward structured systems is not slowing down. AI is reshaping how we impose structure on content. Governments are rewriting rules around data. The archival and records communities are increasingly wrestling with the relational, processual, and algorithmic nature of contemporary evidence.

If you have not read Karl’s original piece, go read it. Then subscribe to his Substack, Meta-IRM. His writing is one of the few places in the field where conceptual clarity meets the practical messiness of daily work.

The structured–unstructured gradient is our real landscape. Not the tidy categories we inherited. Not the fences we built to defend what felt familiar.

The gradient is where evidence lives.
The gradient is where risk resides.
The gradient is where organizations now expect us to lead.

And the gradient is where the future of records management will be decided.

Resources and Further Reading

The ideas in this article draw upon a broad body of research, standards, and professional commentary from across the global records and information management community. The following resources offer deeper insight into the structured–unstructured information gradient, the convergence of records and data management, and the evolving expectations around accountability and governance.

ARMA International

Inglis, Nick. “A Pair of Foundational Concepts.” Information Management Magazine (2019).
A clear explanation of the relationship between content, data, and records and why form does not determine evidential value.

AIIM

AIIM Industry Research Reports (various).
Studies on unstructured information, metadata, and the challenges of legacy file repositories and ROT accumulation.

ISO Standards and Technical Reports

ISO 15489-1:2016. Information and documentation. Records management. Part 1.
The foundational definition of a record as information kept as evidence, regardless of format.

ISO/TR 8344:2024. Issues and considerations for managing records in structured data environments.
A crucial document for understanding how databases, applications, and structured systems generate and maintain records, and how records and data professionals must collaborate to govern them.

Records Continuum Theory

Upward, Frank; McKemmish, Sue; Iacovino, Livia, and others.
A body of work that reconceptualizes recordkeeping as embedded in business processes and systems. This theory supports format-neutral, system-integrated recordkeeping.

Academic Literature

McDonald, John. “Records management and data management: Closing the gap.” Records Management Journal.
Argues that the distinction between records and data management is largely historical and that organizations must govern all information assets holistically.

Kim, Y. “Public Data Management as Records.” Journal of Korean Society of Archives and Records Management.
Explores structured datasets as public records within a global context.

Hamilton, Kelly. “Structured data elements: Are they records?” Information Management 45(2).
A practical exploration of when individual data elements meet the criteria for records and what this means for retention and appraisal.

Information Governance and Data Management

O Brien, Daragh. “Data Is Risky Business: Structured, Unstructured, Who Cares?”
A practitioner-oriented argument that the structured versus unstructured distinction is increasingly obsolete in the age of AI.

Phillips, John. “Structured vs Unstructured Records. Really?” (AIIM Community Blog).
Explains the hidden structure within so-called unstructured content and the narrative context embedded in data.

Government and Regulatory Guidance

United States Federal Records Directives including:

• The Managing Government Records Directive

• OMB/NARA M-19-21
  Affirm digital-first recordkeeping and the inclusion of data as federal records.

European Union General Data Protection Regulation (GDPR)
Reinforces unified governance of personal data across all formats, structured and unstructured.

Digital Preservation and Long-Term Access

Digital Preservation Coalition. “What is a record?”
Details the preservation of content, context, and structure, including for records originating in structured systems.

Professional Practice and White Papers

Wortzmans Legal Analysis (2015). “Information management is not just for unstructured data.”
Shows that structured data environments often accumulate ROT and require rigorous disposition controls.

AccessCorp. “All records are information, but not all information is a record.”
Describes the unification of data governance and records governance within an information lifecycle perspective.