Overview

Two related freelance deliverables for the same client: a fraud-flagging classifier over transaction text, and a sentiment model over customer feedback — both needed to work in English and Persian without maintaining two separate pipelines.

Preprocessing

The same normalization step handles both languages, so downstream models never need to know which language they’re looking at:

def normalize(text, lang):
    text = strip_diacritics(text)
    return tokenizer[lang].tokenize(text.lower())
def normalize(text, lang):
    # multilingual fallback when a language-specific tokenizer isn't available
    return multilingual_tokenizer.tokenize(text.lower())

Result

A shared preprocessing layer plus separate lightweight classifier heads per task kept the system maintainable — new languages could be added by dropping in a tokenizer, without retraining either classifier from scratch.