A brown hat with the text "LDH consulting services" next to it.

Tip of the Hat

19 June 2019

[REDACTED] - Redacting PII From Digital Collections

Welcome to this week’s Tip of the Hat! The Executive Assistant is back, and you know what that means…
A sitting black cat looking up at the camera, meowing loudly.
We’re back in business, newsletter-wise!

This week’s topic comes from a recent post to the Code4Lib mailing list. A library is planning to scan a batch of archival documents to PDF format, and are looking for ways to automate the process of identifying personally identifiable information [PII] in the documents and redacting said PII. The person mentioned that the documents might contain Social Security Numbers or credit card numbers.

Many libraries and archives have resources – digital and physical – that contain some form of PII in the source. While physical resources can be restricted to specific physical locations (unless someone copies the source via copier, pencil and paper, camera, etc.), digital resources that are available through a digital repository can increase the risk of privacy harm if that digital resource contains unredacted PII.

When libraries and archives are incorporating personal collections, research data sets, or other resources that may contain PII, here are some considerations to keep in mind to help through the process of mitigating the risk of data breaches and other privacy harms:

Who is included or mentioned in the resource - Some archival collections contain PII surrounding the individual who donated their materials. When dealing with institutional/educational records or research data sets, however, you might be dealing with different types of PII regulations and policies depending on who is included in the resource and what type of PII is present.

What PII is in the resource – When most folks think about PII, they think about information about a person: name, Social Security number, financial information, addresses, and so on. What tends to be overlooked is PII that is information about an activity surrounding a person that could identify that person. Think library checkout histories, web search histories, and purchase history. You will need to decide what types of PII needs redacting, but keep both facets of PII in mind when deciding.

What is the redaction workflow – This gets into the question from the mailing list. The workflow of redacting PII depends on several factors, including what PII needs to be redacted, the number of resources needing to be redacted, and what format the resource is in. Integrating redaction into a digitization or intake workflow reduces the time spent retroactively redacting PII by staff. Here I’d like to offer a word of caution – while automating workflows for efficiency can be positive, sub-optimizing a part of a workflow can lead to a less efficient overall workflow as well as have negative effects on work quality or resources.

What tools and resources are available – While looking at the overall workflow for redacting PII, the available resources and knowledge available to you as an organization to build and maintain a redaction workflow will greatly shape said workflow, or even the ability to redact PII in a systematic manner. There are many commercial tools that automate data classification and redaction workflows, and there are options to “roll your own” identification and redaction tool using various programming languages and regular expressions. If you work at a library or archive that is part of a bigger institution, there might be tools or resources already available through central IT or through departments that oversee compliance or information security and privacy. Don’t be afraid to reach out to these folks!

If you’re wondering where to begin or what other organizations approach redaction, here are a few resources, here are some resources to start with:

Have a question or topic that you want us to write about? Email us at newsletter@ldhconsultingservices.com!