Profanity filtering identical to better_profanity, up to 12,000× faster

Change one import

The profanity object and the Profanity class take the same arguments as in better_profanity 0.7.0 and return the same results:

from better_profanity_rs import profanity  # was: from better_profanity import profanity

profanity.censor("This sh1t is expensive, you @ss.")
# 'This **** is expensive, you ****.'
profanity.contains_profanity("sh!t happens")
# True
profanity.load_censor_words(["spoiler", "leak"], whitelist_words=["leak"])
profanity.censor("No spo1ler and no leak, please.", "-")
# 'No ---- and no leak, please.'

load_censor_words_from_file, add_censor_words and the attributes CENSOR_WORDSET, CHARS_MAPPING, MAX_NUMBER_COMBINATIONS and ALLOWED_CHARACTERS work as before.

Speed up code that imports better_profanity

When a library or an app module imports better_profanity itself, call install() first:

import better_profanity_rs

better_profanity_rs.install()  # `import better_profanity` now returns better_profanity_rs

import my_web_app

Modules that imported better_profanity before the call keep the original.

How much faster

contains_profanity on windows of Jigsaw toxic comments, median of 20 windows per length. Apple M1 Pro, CPython 3.13; censor gives the same figures.

The whole civil_comments test split, 97,320 comments and 5.0 million words, takes 1.10 s. Method and raw results: BENCHMARKS.md.

Why it is so much faster, and still exact

better_profanity keeps its 916 words in a Python list. Every word of the text, alone and joined with the words after it, is compared with each of them by a method that tries the leetspeak variants letter by letter, so a 100-word message costs close to a million comparisons.

Every leetspeak variant is a single character, so a word is a sequence of character sets (a, @, * and 4 for an a). better-profanity-rs stores the list as a trie with character-set edges and runs the original scan, step for step, in Rust: the same word windows, separators, whitelist checks and replacement rules, including the original's quirks.

How "identical" is checked

The tests compare every output and exception with better_profanity 0.7.0 on CPython 3.9 to 3.14: its own test suite, leetspeak words inside texts and joined by separators, custom word lists and whitelists, invalid arguments, attribute changes, subclasses, threads and property-based tests. A separate script compares 313,257 texts, among them the civil_comments test split and the Jigsaw toxic comments, with the default and a custom word list on four Python versions, and a workflow repeats it on Linux, macOS and Windows: no output differs.

For anything the Rust core cannot reproduce exactly, better_profanity's own code, bundled in the package, runs instead, so the output stays the same:

• text that is not an exact str or contains lone surrogates;
• a word list that is not a list or tuple of the original's word objects, or CHARS_MAPPING variants longer or shorter than one character;
• a replaced ALLOWED_CHARACTERS, patched helper methods, and calls made close to the recursion limit.

Known differences, such as changes to private attributes of words already in the list, are listed in the README. Free-threaded CPython builds are not supported.

Rust

use better_profanity::Profanity;

let profanity = Profanity::new();
assert_eq!(
    profanity.censor("That wh0re gave m3 a very good H@nD j0b.", "*"),
    "That **** gave m3 a very good ****."
);

let mut custom = Profanity::with_words(["happy", "jolly"]);
custom.add_censor_words(["merry christmas"]);
assert!(custom.contains_profanity("Merry Christmas!"));

The crate lowercases with Rust's Unicode tables; the Python package uses the interpreter's str.lower(), as better_profanity does.