Vol. IX · Data Plate 6 of 12PII, personal information, redaction, toxicity filtering, blocklists, false positives, C4DE

Safety and Privacy Filters.

Find the e-mail addresses, mask the phone numbers — and notice how many medical and technical pages a word list throws away.

A web crawl contains personal information: e-mail addresses, phone numbers, home addresses, sometimes identity or card numbers. A model that reads them can memorise and repeat them (plate VIII.7), so pipelines detect and mask them, usually with pattern matchers — regular expressions for the common formats — and sometimes with trained recognisers for names and addresses. Masking replaces the match with a placeholder such as [EMAIL], so the surrounding text stays useful.

Harmful content is harder, because harm is not a pattern. The simplest approach is a blocklist: drop any document containing a listed word. The C4 dataset did exactly this, and an audit by Dodge and colleagues found that it removed disproportionately much text by and about minority groups, and medical and other legitimate text, while leaving much objectionable text in. Trained classifiers do better but share the trade-off: every threshold removes some innocent text and keeps some harmful text. The workshop measures both on the course’s own writing.

Below: left, a text with personal information detected and masked — edit it to test the patterns. Right, course paragraphs that a naive word list flags, with the word that triggered each.

What you will learn
Workshop
Left: regular-expression detectors for e-mail addresses, phone numbers and card-like numbers applied to the typed text, with matches masked. Right: the longer course paragraphs checked against an eight-word list (kill, attack, drug, shot, breast, strike, abuse, weapon); every paragraph is educational, so every flag is a false positive.

Every flag on the course is a mistake

The patterns find 3 items in the default text — an e-mail address, a phone number and a card-like number — and mask each with a placeholder, leaving the sentence usable. They miss the name, which needs a trained recogniser, and they would also mask any long product code that happens to look like a card. The word list does worse. It flags 3 of 61 course paragraphs, a false-positive rate of 5%: “abuse” is mathematicians’ slang for loose notation, “attack” is an acid attack on teeth or an immune attack, “drug” is pharmacology. Every flag is wrong, because the corpus is a set of science courses.

At web scale the same mistakes become a bias. Dodge and colleagues found that C4’s blocklist removed text about health, sexuality and minority identities at higher rates, so a model trained on it saw less of those subjects and of the dialects used to discuss them. Better classifiers reduce but do not remove the trade-off, and many teams now filter only the most clearly harmful material at pretraining, leaving behaviour to be shaped later (Vol. XX), where context can be taken into account.

Masking is not the same as removal: a placeholder keeps the document’s structure and teaches the model that such slots exist, without the actual address. Some pipelines go further and replace personal data with realistic fakes.

Terms

Personal information (PII)
Data that identifies a person: names, addresses, e-mail, phone, identity numbers.
Masking
Replacing a detected item with a placeholder such as [EMAIL].
Blocklist
A list of words whose presence removes a document.
False positive
Harmless text that a filter flags.
Pattern matchers mask personal data cheaply; word lists flag innocent medical and technical text, and that over-filtering falls unevenly.

Sources

  1. Dodge et al., “Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus”, EMNLP 2021.
  2. Raffel et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer”, JMLR 2020, section 2.2 (C4’s word-list filter).