Detecting Data Contamination in Marketing Language Models: A Domain-Aware Research Protocol

1. Introduction and research question

Marketing language models are increasingly trained and evaluated on text that is both commercially consequential and structurally repetitive. Customer-service scripts, influencer captions, promotional responses, compliance templates, and campaign summaries often reuse the same phrases across channels, brands, and time. That repetition is useful for operational consistency, but it also creates a contamination risk when the same or closely related text enters both training and evaluation corpora. In that setting, a model may appear to perform well because it has seen the target patterns before rather than because it has learned a transferable representation of marketing language. The present article asks a bounded methodological question: how should contamination be detected and reported in marketing-language model studies when the corpus includes multiple text types, multilingual content, and repeated promotional formulas? [1] [2] [3]

The evidence scope is a targeted narrative synthesis rather than a systematic review. The selected sources are used in two tiers. First, direct application evidence from marketing-language settings shows that domain text in finance, customer support, influencer communication, retail promotion, and pharmaceutical marketing is linguistically specialized and operationally diverse. Second, methodological foundation from domain-aware automated machine learning shows that marketing analytics benefits from explicit feature engineering and pipeline control. Together, these sources justify a contamination protocol that is sensitive to domain vocabulary, text reuse, and workflow boundaries, while recognizing that none of them directly establish leakage tests for marketing language models. [4] [1] [2] [3] [5] [6]

2. Evidence base and problem definition

The source literature indicates three relevant properties of marketing-language data. First, domain expressions are not interchangeable with general-purpose language. In multilingual financial sentiment analysis, model behavior depends on domain-specific expressions and cross-lingual representation, which means that contamination checks must capture not only exact duplicates but also translated or paraphrased reuse across languages. Second, conversational marketing systems can rely on structured query-to-response mappings and knowledge-graph representations, which makes them vulnerable to prompt, schema, or response-template leakage. Third, promotional and retail language often uses repeated persuasive formulas, so a naive duplicate filter may remove benign routine content while missing semantically equivalent contamination. These properties imply that contamination detection must be both lexical and semantic, and must be calibrated to the task type rather than applied as a generic text deduplication step. [4] [1] [2] [3]

For this protocol, contamination is defined as any overlap between training, validation, test, or prompt-adaptation material that can reasonably permit direct memorization or shortcut inference. The definition includes exact duplication, near duplication after normalization, translated duplication across languages, label-bearing examples reused in prompt or fine-tuning data, and reused instruction templates that preserve task-specific cues. This definition is intentionally broader than duplicate sentence matching, because marketing systems often encode the same campaign logic in multiple surface forms. At the same time, it is narrower than a general notion of data quality: unrelated noise, disagreement, or annotation error are not contamination unless they create leakage across data partitions. [4] [1] [6]

3. Conceptual foundations from prior work

The proposed protocol draws its methodological rationale from the way prior work constructs and evaluates marketing language systems. EMSA for multilingual sentiment analysis uses prompt engineering and explicit reasoning steps to handle cross-lingual domain text, showing that evaluation must account for language-specific expressions and domain specificity. That lesson transfers directly to contamination detection: if a marketing corpus mixes languages or transliterated terms, a detector should compare semantic similarity after normalization, translation, and domain-term mapping rather than rely only on string equality. [4]

The iDigiChat study is relevant because it turns customer queries into structured graph representations and processes them through a knowledge-graph-plus-neural-network pipeline. This illustrates that marketing language data may be stored or generated in semi-structured forms, such as intents, slots, labels, and response templates. Contamination detection in such settings must therefore inspect both free text and structured metadata. If the same intent patterns or answer graphs appear in both training and evaluation sets, a model can exploit schema familiarity even when surface text differs. [1]

Studies of influencer marketing and beverage retail show that promotional language is not random prose. It is organized around repeated cues such as trust, emotional connection, brand awareness, discounts, and placement effects. These recurring factors produce high-frequency lexical patterns that can obscure leakage because similarity may reflect genuine marketing conventions rather than copied documents. A contamination detector must therefore estimate whether repeated language is expected for the task or whether it carries document-level dependency across splits. This distinction matters in retail audits and influencer analyses alike, where repeated message structure is common but data partitions should still remain independent. [2] [3]

The AutoML foundation is relevant for a different reason. Marketing-AutoM3L emphasizes domain-aware feature engineering and pipeline assembly, rather than generic tabular automation. The contamination protocol follows the same design principle: the detector should be pipeline-aware, with explicit steps for data modality recognition, feature extraction, similarity scoring, and decision rules. In other words, contamination detection is treated as a reproducible pipeline, not as a single heuristic. [6]

4. Proposed methodology: variables, operational definitions, and detection logic

The unit of analysis is a record pair across partitions, where each record may be a document, utterance, prompt-response exchange, or structured marketing message. The main independent variables are lexical overlap, semantic overlap, source provenance overlap, and template overlap. Lexical overlap is computed after normalization that lowercases text, removes punctuation, standardizes numerals, and applies domain-term canonicalization. Semantic overlap is estimated through embedding similarity and cross-lingual alignment where applicable. Source provenance overlap is a binary indicator that the two records originate from the same campaign, vendor, customer account, or time window that the study has declared non-independent. Template overlap captures repeated instruction patterns, slot structures, or response frames. [1] [4] [6]

An unambiguous default rule is as follows. For each cross-partition pair \(i,j\), define \(J_{ij}\) as normalized token-level Jaccard similarity, \(S_{ij}\) as embedding cosine similarity after domain calibration, \(P_{ij}\in\{0,1\}\) as the provenance-overlap indicator, \(L_{ij}\in\{0,1\}\) as the example-specific answer leakage indicator, excluding mere reuse of the label ontology, \(R_{ij}\in\{0,1\}\) as the template-overlap indicator for prompt or response structure, and \(X_{ij}\in\{0,1\}\) as the cross-lingual reuse indicator. To set \(X_{ij}=1\), the protocol applies a reproducible two-step rule only to pairs whose dominant languages differ. First, both records are mapped to a shared multilingual embedding space and their cosine similarity \(M_{ij}\) is computed. Second, each record is translated into the other record’s language with one frozen translation system chosen before evaluation, and a translation-normalized lexical similarity \(J^{tr}_{ij}\) is computed after the same normalization used for \(J_{ij}\). Cross-lingual reuse is flagged when \(M_{ij}\geq T_M\) and \(J^{tr}_{ij}\geq T_X\). A pair is then flagged as contaminated if \(L_{ij}=1\), or if \((P_{ij}=1)\land[(J_{ij}\geq T_J)\lor(S_{ij}\geq T_S)\lor(X_{ij}=1)]\). This precedence is intentional: label leakage alone is sufficient for a contamination flag, whereas lexical, semantic, or cross-lingual similarity requires overlap across a declared non-independent source boundary. For prompt-based tasks, a prespecified sensitivity analysis may apply the stricter rule \((P_{ij}=1)\land R_{ij}=1\land[(J_{ij}\geq T_J)\lor(S_{ij}\geq T_S)\lor(X_{ij}=1)]\), but the preregistered default should remain the rule above unless changed before any final evaluation. [4] [1]

The decision sequence is: (1) ingest all corpus partitions and metadata; (2) normalize text and canonicalize domain terms; (3) detect exact duplicates by hashing; (4) compute near-duplicate lexical similarity; (5) compute semantic similarity within and across languages; (6) detect template reuse in prompts, instructions, and response skeletons; (7) apply the pairwise contamination rule; and (8) aggregate pairwise flags to record-level and partition-level decisions. Aggregation is defined prospectively. A record receives a candidate audit flag only for a prohibited cross-role pair under the registered partition policy. Quarantine requires direct adjudication of equivalent-example or example-specific answer leakage; generic template similarity alone never propagates quarantine transitively. A partition-level contamination burden for partition \(k\) is then \(B_k=n_k^{-1}\sum_{r\in k} I_r\), where \(I_r=1\) if record \(r\) is marked contaminated and \(n_k\) is the number of records in that partition. In parallel, each pair receives a continuous risk score \(Q_{ij}=w_J\tilde J_{ij}+w_S\tilde S_{ij}+w_P P_{ij}+w_R R_{ij}+w_L L_{ij}+w_X X_{ij}\), where the non-binary similarity inputs are rescaled to \([0,1]\), all weights are constrained to be non-negative, and \(\sum w=1\). The binary contamination flag is generated by the explicit rule in the previous paragraph; the risk score is used only for ranking, audit review, and calibration analysis. To avoid circular tuning, parameter selection should use nested development resampling or a fixed three-way development split that is disjoint from the final test set. Under the default three-way design, development subset A is used only to choose normalization, embeddings, translation settings, and template parsers; subset B is used only to tune \(T_J\), \(T_S\), \(T_M\), and \(T_X\) for the binary rule; subset C is used only to estimate the weights for \(Q_{ij}\) and to assess calibration of the continuous score. If cross-fitting is preferred because development data are limited, each stage must still be estimated on folds not used to evaluate that same stage. After these steps, all parameters are frozen before any inspection of the held-out test partition. A suggested comparator is an exact normalized-text hash baseline, with a separately reported MinHash/shingle-similarity baseline for near duplicates; an exact hash cannot detect approximate reuse. [1] [6]

The protocol assumes that the investigator has access to partition metadata and can distinguish independent from non-independent sources. If provenance metadata are incomplete, the detector shifts to conservative mode and treats repeated campaign wording as suspicious unless independently justified. This assumption is important in marketing corpora because campaigns are frequently syndicated across channels. It is also a limitation: without provenance, the detector may over-flag legitimate reuse in brand-consistent copy. [2] [3]

5. Validation design and outcome measures

Because this is a protocol rather than a results report, the validation design is prospective. The primary validation corpus should contain held-out marketing texts from at least three operational settings represented in the evidence base: customer-service dialogue, promotional and influencer language, and retail or workflow communication. The corpus should be partitioned by source entity and time to reduce hidden overlap. Synthetic contamination should then be injected under a prespecified stratified allocation scheme. A default design is to reserve a clean control subset with no injected contamination and to inject the remaining contaminated cases evenly across validation and test partitions, with equal counts for five modes: exact duplication, paraphrase, cross-lingual translation reuse, template reuse, and label leakage. Injection should be stratified by operational setting and by partition so that each mode appears in each setting where it is substantively plausible, while source and time labels are preserved to keep provenance constraints meaningful. This design allows the detector to be tested against contamination modes that are common in marketing language systems but not fully observable in natural corpora. [1] [2] [3] [5]

Outcome measures should include pair-level precision, recall, and F1 for contaminated-pair identification; record-level contamination burden within each partition; and the change in downstream model performance after quarantine, if a downstream model evaluation is later conducted. For this conceptual protocol, the primary planned outcomes are detector accuracy and review burden rather than any unperformed model-performance gain. A useful secondary measure is calibration error for the continuous risk score, defined as the discrepancy between predicted risk and observed contamination frequency across prespecified bins on validation data that were not used to estimate the score weights. Another outcome is review burden, measured as the number or proportion of records quarantined per true contaminated record detected. These measures should be reported separately for exact-duplicate, paraphrase, cross-lingual, template-reuse, and label-leakage contamination modes, because the protocol is intended to detect multiple leakage mechanisms rather than a single duplicate class. [2] [3]

Comparative evaluation should include at least three baselines: exact-duplicate hashing, sentence-embedding nearest-neighbor screening, and a rule-based provenance filter. The proposed method is expected to outperform exact duplicate filtering on paraphrase and translation leakage, and to outperform pure embedding screening on template and provenance leakage. However, no empirical gain should be claimed until tested. The objective is to establish a testable design in which improvements can be attributed to specific components of the detector rather than to opaque preprocessing. [4] [1] [6]

6. Failure cases and boundary conditions

The first failure case is paraphrased leakage. Marketing language often preserves meaning while changing wording, especially in promotional copy and customer support responses. A detector that depends on exact n-gram overlap will miss this. A detector that depends only on semantic similarity may overflag legitimate topical similarity, especially when many documents discuss the same product or campaign. The protocol therefore uses a combined lexical-semantic rule rather than either criterion alone. [1] [2]

The second failure case is cross-lingual contamination. Multilingual marketing corpora may contain equivalent content in different languages, or partially translated customer-service content. Prior multilingual sentiment work indicates that domain-specific meaning can be preserved across languages while surface forms diverge, which makes translation-normalized comparison necessary. Yet translation itself can blur distinctions, so the method should record whether similarity is language-paired or translation-mediated rather than collapsing them into one score. [4]

The third failure case is legitimate repetition. Brand messaging, product disclaimers, and standard response scripts may be highly repetitive without implying contamination. This is especially true in retail promotions and customer-service workflows. To limit false positives, the protocol treats repetition as suspicious only when it crosses declared independence boundaries or when the same label-bearing instance appears in both training and evaluation partitions. That rule is strict enough for evaluation integrity but may be too conservative for corpus cleaning aimed at downstream retrieval tasks. [1] [3]

The fourth boundary condition concerns workflow transfer. AI adoption studies in pharmaceutical marketing suggest that organizational readiness, governance concerns, and role-specific use shape how AI is deployed in practice. This implies that contamination controls may be easier to implement in centralized marketing teams than in fragmented multi-vendor ecosystems. The protocol generalizes best where source control is available; it is less stable in open social-media scraping or in settings where synthetic and human-authored content are mixed without provenance labels. [5]

7. Relation to specific prior methods

Compared with EMSA, the proposed protocol does not attempt sentiment classification or interpretive reasoning; it borrows the methodological insight that domain-specific language demands explicit control of language and task context. Its contribution is narrower and more defensive: it seeks to prevent evaluation inflation before model comparison begins. Unlike EMSA’s prompt-based classification, contamination detection here is a preprocessing and audit function. [4]

Compared with iDigiChat, the protocol does not build a customer-service response engine or a knowledge graph. It uses the same structural insight that marketing language can be represented in modular form and that template structure matters. The difference is that iDigiChat aims to improve service response, whereas this protocol aims to detect whether such responses, or their templates, recur across partitions in a way that compromises evaluation. [1]

Compared with the influencer-marketing model, the present protocol does not test consumer perception or purchase intention. It adopts the observation that marketing language contains stable cues such as trust, brand awareness, and emotional appeal, which can produce dense semantic similarity. That observation motivates a detector that does not confuse legitimate topical clustering with leakage. The distinction is important because a high-similarity neighborhood in promotional text may reflect genre regularity rather than contamination. [2]

Compared with the beverage retail audit, this protocol moves from observing marketing conduct to auditing dataset integrity. The retail study shows that promotional behavior can be measured across time and store context; the present proposal adapts that logic to text corpora by measuring overlap across source and time partitions. Compared with the AI-adoption study, the proposal does not analyze user acceptance, but it adopts the same principle that governance and workflow conditions shape whether a technical method can be used responsibly. Compared with Marketing-AutoM3L, it extends domain-aware automation from feature engineering to data-integrity control. [3] [5] [6]

Partition roles and contamination challenge construction

Freeze a role matrix before matching: training and prompt-adaptation overlap may be permitted when declared, while a final held-out test cannot appear in either. Within-partition redundancy is a separate diagnostic. Shared class names do not constitute leakage; the detector must identify an example-associated answer or equivalent task instance. Soft labels, multiple labels and free-form responses require task-specific equivalence rules and adjudication. Construct clean source-group and time-separated partitions first, then inject known leaks using disjoint anchor records for validation and test. Preserve every injection’s synthetic provenance and leave clean controls unmodified. Evaluate a registered prevalence grid as well as any balanced challenge set; balanced precision is not deployment precision. Risk scores rank review and do not certify contamination. A missing provenance field warrants unresolved review rather than an asserted violation. Report confirmed contamination separately from candidate burden, including empty-partition metrics as non-estimable. No end-to-end leakage detector has been validated in this article.

8. Conclusion

This manuscript establishes a conceptual and procedural framework for detecting contamination in marketing language models. The core claim is that contamination in this domain should be defined through a combined assessment of lexical duplication, semantic similarity, provenance overlap, and template reuse, with separate attention to multilingual reuse and structured conversational patterns. The protocol is grounded in marketing-language evidence showing that domain text is repetitive, cross-lingual, and workflow-specific, and in methodological evidence showing that marketing analytics benefits from domain-aware pipelines rather than generic automation. The proposal is intentionally conservative: it favors auditability and falsifiability over broad claims of novelty or performance. [4] [1] [2] [6] [3] [5]

What remains untested is whether the proposed detection sequence yields better contamination recall with acceptable false-positive rates across real marketing corpora, especially when provenance is incomplete and paraphrase is extensive. That question requires a prospective evaluation with held-out partitions, synthetic contamination injection, and comparison against exact-match and embedding-only baselines. Until such testing is performed, the protocol should be treated as a precise methodological proposal for improving evaluation integrity in marketing language model research, not as a validated benchmark. [1] [4] [6]

References

1. He, Ji; Luo, Yi; Wang, Ting (2025). iDigiChat: intelligent digital marketing service chatbot for providing efficient customer services using artificial intelligence. Scientific Reports. 15(1), 33074. doi:10.1038/s41598-025-14722-5.
2. Babu, Md. Asaduzzaman; Urmi, Hafsa Akter; Tofayel, Abzal Hosen; Sabuj, Md. Mehedul Islam; Alam, Md Tamjid Ul (2025). The impact of influencer marketing in the tourism industry: A digital marketing perspective. PLOS One. 20(12), e0338423. doi:10.1371/journal.pone.0338423.
3. Dooley, Kierra; Zaltz, Daniel A.; Harding, Scott V.; Crichton, Kayla; Hammond, David; Yi, Yanqing; Paquette, Marie-Claude; Wang, Peizhong Peter; et al. (2026). In-store beverage pricing and marketing before and after a sugar-sweetened beverage tax in Newfoundland and Labrador. Public Health Nutrition. 29(1), e58. doi:10.1017/s1368980026102146.
4. Zhao, Li; Zhou, Jinwei; Cao, Jinde; Zhu, Weina (2025). EMSA: Explainable multilingual sentiment analysis models providing sentiment analysis across multiple languages. PLOS One. 20(11), e0333508. doi:10.1371/journal.pone.0333508.
5. Boruga, Veronica Madalina; Bratu, Melania Lavinia; Puenea, George; Popa, Daniel; Popa, Cristina Annemari; Bogdan, Iulia Georgiana; Savencu, Cristina Elena (2026). Organizational Readiness, Perceived Usefulness, and Determinants of Artificial Intelligence Adoption in Romanian Medical Management and Pharmaceutical Marketing. Healthcare. 14(12), 1714. doi:10.3390/healthcare14121714.
6. Tian, Ye; Shao, Wenqian; Deng, Zihan (2026). Marketing-AutoM3L: domain-aware automated machine learning for financial customer analytics. Frontiers in Artificial Intelligence. 9, 1726900. doi:10.3389/frai.2026.1726900.