Abstract
Influencer selection models are often assessed by predictive fit, yet their recommendations may rest on evidence that is narrow, context-bound, or opaque. This conceptual manuscript proposes an audit framework for evidence quality in influencer selection and prediction models, using tourism influencer marketing as a concrete domain while remaining methodologically transferable. The research question is whether a selection model can be audited not only for statistical performance but also for the quality, traceability, and task alignment of the evidence on which its recommendation depends. The proposed framework integrates five methodological foundations: systematic mapping of the evidence base, randomized experimental logic for causal claims, explainability of model outputs, algorithm-to-task benchmarking, and comparative forecasting evaluation. The design is intentionally framed as a targeted narrative synthesis rather than a systematic review, because the objective is to define an auditable decision sequence, not to estimate literature prevalence. The core contribution is an operational protocol that converts heterogeneous evidence into a structured audit record with explicit rules for source weighting, comparator choice, and failure diagnosis. A central rule is that a model should be flagged as evidence-weak when its recommendation changes under leave-one-source-out perturbation by more than a prespecified hypothetical threshold, or when its explanatory features are not causally interpretable for the claim being made. The manuscript specifies variables, decision steps, comparator classes, validation tests, boundary conditions, and limits of generalization. It therefore offers a precise basis for future empirical evaluation of influencer selection systems without claiming completed testing.
1. Introduction and Research Question
Influencer selection systems are increasingly used to rank or recommend social media accounts for marketing collaboration, yet their outputs are commonly treated as if a high predictive score implies trustworthy evidence. That inference is too strong. A model may predict engagement, purchase intention, or perceived credibility well while still relying on weak proxies, context-specific associations, or untested assumptions about causality. In tourism marketing, for example, prior work shows that perceived trust, content quality, word of mouth, emotional connection, and brand awareness are associated with purchase intention, and that the relative importance of these factors can vary by context such as Bangladesh and by the distinction between human and virtual influencers [Babu et al.]. This provides a useful domain setting for asking a broader methodological question: how should one audit the evidence quality behind a model that selects influencers or ranks influencer candidates? [1]
The research question addressed here is: What audit procedure can determine whether an influencer selection model is evidence-adequate for a specified decision task? Evidence-adequate means that the model’s inputs, training evidence, output explanation, and validation design are all aligned with the claim it makes. The claim may be descriptive, predictive, or causal, but the audit must distinguish among these types rather than allow them to collapse into a single performance score. The proposed framework is conceptual and prescriptive. It does not report new experiments or a completed review. Instead, it defines a targeted narrative synthesis and an operational audit protocol that can be applied to a set of candidate models, datasets, and source studies in a bounded domain such as tourism influencer marketing.
2. Evidence Scope and Synthesis Strategy
The evidence scope is deliberately narrow. The direct application source is used only to anchor the domain problem of influencer marketing in tourism; it is not treated as evidence that any particular audit method has already been validated for influencer selection. The remaining sources provide transferable methodological foundations. A systematic map of climate-risk valuation studies shows how evidence can be coded by risk type, asset class, geography, and method in order to reveal coverage gaps and methodological concentration [Piseddu et al.]. An indoor air quality machine-learning review demonstrates the value of mapping data structure to algorithm family and of noting that no universal algorithm exists across tasks [Rosca et al.]. An explainable multilingual sentiment analysis study illustrates how model outputs can be paired with explicit reasoning and compared to strong baselines rather than assessed only by raw accuracy [Zhao et al.]. A randomized experiment on price disclosure in pharmaceutical advertising shows how causal claims require explicit experimental contrasts and outcome definitions [Hollin et al.]. A forecasting study using U-MIDAS emphasizes comparator choice and forecast-error metrics when the task is prediction under mixed-frequency data [Singh et al.]. [2] [3] [4] [5] [6]
The synthesis strategy is targeted narrative synthesis. It asks what each source contributes to an audit framework, what assumptions it supports, and where its method should not be extended. It does not attempt a frequency estimate of methods in the literature, a formal meta-analysis, or a reproducible database search. That choice is appropriate because the manuscript’s purpose is to define a rigorous audit architecture rather than to claim prevalence. The evidence base is therefore treated as a set of methodological exemplars from adjacent domains, plus one application context from influencer marketing. The proposed framework remains agnostic about whether the future user is auditing a recommender system, a supervised ranking model, or a human-in-the-loop selection process.
3. Audit Variables and Operational Definitions
The audit uses four classes of variables. First, task variables define what the model is supposed to do. These include the decision target T, such as predicted engagement, predicted conversion likelihood, or predicted fit between influencer and campaign. Second, evidence variables describe the sources on which the model depends: source type S_i, domain similarity D_i, recency R_i, and causal strength C_i. Third, output variables describe the model recommendation: rank position, selection probability, feature attribution, and confidence interval when available. Fourth, audit variables describe the quality of the evidence chain: traceability, causal validity, calibration, and robustness under perturbation.
Operational definitions are needed to prevent category confusion. Domain similarity D_i is the degree to which the source evidence was generated in the same market, platform ecology, and decision setting as the target task. Causal strength C_i is the degree to which a source supports a causal claim rather than a mere association. Explainability score E is the fraction of the top-k features in the model explanation that can be linked to evidence appropriate for the claim type. For a predictive claim, associative evidence may suffice; for a causal claim, it does not. A simple operational rule is as follows: if a model’s explanation for an influencer recommendation relies on features f_1...f_k, then E = m/k, where m is the number of features that are directly supported by task-relevant evidence. If E < 0.6, the output is flagged as explanation-weak under a hypothetical audit threshold. This threshold is proposed for illustration only and must be tuned to the application domain.
4. Proposed Audit Architecture
The audit proceeds in six steps. Step 1, claim typing: classify the model’s stated use into descriptive ranking, prediction, or causal recommendation. Step 2, evidence inventory: enumerate training data, feature sources, human labels, and external studies or rules used in feature selection. Step 3, evidence coding: assign each source evidence item values for domain similarity, recency, and causal strength. Step 4, task-model alignment: match the algorithm family to the data structure and target variable. This step follows the logic that model choice should be driven by task structure rather than popularity; the IAQ review is explicit that no universal algorithm exists and that data quality and structure shape success [Rosca et al.]. Step 5, explanation audit: assess whether the model’s output explanation makes a claim stronger than the evidence warrants. Step 6, robustness audit: perturb the evidence base and observe whether the recommendation is stable. [3]
The comparator is not a single rival model but a class of reference conditions. For a ranking model, the comparator may be a simple baseline such as rank-by-popularity, rank-by-engagement, or rank-by-trust proxy. For a forecasting model, the comparator should include an alternative specification with the same prediction target but different modeling assumptions, following the practice of comparing U-MIDAS with ARIMAX under forecast-error metrics [Singh et al.]. For a causal claim, the comparator should be an experimental or quasi-experimental design with explicit treatment and control conditions, consistent with randomized experiment logic [Hollin et al.]. The audit does not require all three comparator types in every case; it requires the comparator to be structurally appropriate to the claim type. [6] [5]
A practical decision sequence is therefore: classify claim; inventory evidence; compute evidence scores; evaluate algorithm-task fit; compare against the correct baseline; test stability under source removal; and assign an audit status. Status categories are proposed as evidence-adequate, evidence-conditional, and evidence-weak. Evidence-conditional means the model may be useful for exploratory ranking but should not support a causal or high-stakes decision claim. Evidence-weak means the recommendation depends on evidence that does not satisfy the minimum alignment needed for the stated use.
5. How the Framework Relates to Prior Methods
The proposed audit framework borrows structure, not conclusions, from the supplied methodological foundations. The systematic map of climate-risk valuation studies is relevant because it codes evidence by dimensions that matter for transferability: risk type, geography, asset class, data source, and method [Piseddu et al.]. The present framework adapts that logic by coding influencer evidence by platform, audience segment, market type, and claim strength. The key difference is that the map is an evidence-survey tool, whereas the audit is a decision-control tool. It asks not only what evidence exists, but whether the model’s recommendation can legitimately depend on it. [2]
The randomized pharmaceutical advertising experiment is relevant because it makes clear that causal claims require explicit exposure conditions and outcome measures, and that behavioral intentions may not move even when perceived information quality does [Hollin et al.]. For influencer selection, this means a model should not claim that an influencer causes purchase intention unless the evidence base includes causal identification, not only correlations between influencer features and observed outcomes. A selection model trained on observational engagement data may still be useful, but only for predictive ranking, not causal endorsement. This boundary is central to the audit and cannot be inferred from fit metrics alone. [5]
The explainable sentiment analysis framework is relevant because it pairs performance comparison with transparent reasoning steps and baseline comparison [Zhao et al.]. The contribution of that study is methodologically transferable: explanation is treated as part of system evaluation rather than a decorative add-on. In the present framework, feature attribution is accepted only if it is compatible with the claim type and if the explanation can be read against the evidence coding. A model that identifies authenticity, narrative consistency, or content quality may be reasonable for tourism influencer ranking, but those features must be linked to evidence of the appropriate scope; otherwise the explanation overstates what the data can support. The sentiment study also shows the importance of checking where prompt-based or reasoning-based methods fail, which motivates the failure-case analysis below. [4]
The U-MIDAS housing-price study provides a model of comparator discipline and forecast evaluation. It shows that mixed-frequency prediction requires choosing metrics that correspond to the prediction target and comparing against a simpler benchmark [Singh et al.]. In the audit framework, an influencer selection model that claims to predict campaign outcomes should be evaluated with outcome-congruent metrics such as rank correlation, calibration, Brier-type loss, or top-k hit rate, depending on the exact target. If the model is used for short-horizon forecasting of expected performance, it should be compared with simpler forecasting baselines rather than only with its internal ablations. [6]
6. Validation Design and Outcome Measures
The prospective validation design has two layers. The first layer is retrospective and documentary: inspect the model, dataset, and evidence base, then record audit variables. The second layer is prospective and task-based: apply the model to a held-out set of influencer-campaign cases or to future campaign opportunities, then evaluate whether the recommendation quality matches the intended claim. The design can be implemented without large samples, provided that the test set is independent and the claim type is explicit. Because this article is conceptual, the validation design is specified but not executed.
Outcome measures are defined as follows. Predictive outcome: ΔP, the difference between model performance and baseline performance on the same task. Explainability outcome: the proportion of explained decision weight that maps to evidence-supported features. Robustness outcome: the change in ranking when evidence items are removed one at a time; a large change indicates dependence on a brittle evidence source. Causal-claim outcome: whether the recommendation remains stable under a design that separates exposure from outcome measurement, such as randomized assignment to different influencer-message conditions. A model is audit-failing if it claims causality but only predictive evidence is available, even if ΔP is positive.
A mathematical decision rule can be stated in plain text as follows: let R be the model’s ranking over candidates and let R(-i) be the ranking after removing source i from the evidence base. Define instability I = average over i of Kendall distance between R and R(-i). If I exceeds a hypothetical threshold τ, then the model is labeled evidence-fragile. This rule is useful because it tests whether a recommendation depends on a small number of unverified sources. The threshold τ should be set prospectively and may differ by domain; no universal value is asserted here.
7. Failure Cases, Boundary Conditions, and Limits of Generalization
Several failure cases are foreseeable. First, a model may use strong predictive features that are weakly interpretable, such as platform-specific engagement traces, and thus may perform well while remaining unsuitable for a causal claim. Second, a model may be well explained but poorly calibrated because its explanation uses plausible features that are not stable across audiences or campaign types. Third, the model may be evidence-rich but task-misaligned: for example, it may learn from fashion or beauty influencer studies and then be applied to tourism without sufficient domain similarity. The tourism source illustrates why this matters: travel influencer effects depend on trust, authenticity, and destination context, and those factors may not transfer unchanged across sectors [Babu et al.]. [1]
Boundary conditions are equally important. The framework is most suitable when the model’s claim can be typed clearly and when the evidence base can be encoded at the source level. It is less suitable for purely creative recommendation systems whose output is not intended to make testable claims about audience response. It is also limited when the available evidence is all observational and the intended use is causal; in that case the audit can identify the mismatch but cannot repair it. Generalization beyond influencer selection is possible only at the level of method: the same audit logic may apply to candidate-ranking systems in hiring, medicine, or education, but the evidence variables and task comparators would need to be redefined for those settings.
The framework also has a temporal limit. Evidence quality may change as platforms, algorithms, and audience norms shift. Therefore, an audit should not be treated as a one-time certification. It should be versioned alongside the model and the evidence corpus. This is consistent with the systematic map logic that evidence bases are shaped by time, geography, and method rather than fixed once and for all [Piseddu et al.]. [2]
8. Conclusion
This manuscript established a conceptual answer to a narrow but practical question: how can influencer selection models be audited for evidence quality rather than only for statistical fit? The proposed framework defines a claim-typed, source-coded, comparator-sensitive audit procedure that distinguishes predictive usefulness from causal adequacy. It also specifies an operational rule for explanation quality and a perturbation-based stability check for evidence fragility. By combining evidence mapping, randomized-experiment logic, explainability, algorithm-task benchmarking, and forecast comparison, the framework converts heterogeneous evidence into a structured audit record.
What remains untested is empirical effectiveness. The manuscript does not claim that the thresholds proposed here are optimal, that the audit scales cleanly across all platforms, or that the framework improves decisions in practice. Those questions require prospective application to real influencer selection cases, ideally across more than one domain and with explicit pre-registration of claim type and comparator. The contribution of this article is therefore bounded: it offers a precise conceptual protocol for auditing evidence quality in influencer selection models, together with a falsifiable set of validation steps for future work.
Contribution and availability
This conceptual manuscript defines an auditable framework for influencer selection models that separates predictive performance from evidential adequacy. Its main contribution is a precise decision protocol with operational variables, claim typing, comparator selection, explanation checks, and perturbation-based stability testing, adapted to influencer marketing while remaining transferable to other selection and prediction tasks.
Funding: Roman Vlasov’s own funds. Editorial responsibility and competing interests: Roman Vlasov is the founder and Editor-in-Chief of Smart World Journal and the creator of the BILDRY construction-planning platform. Other competing interests have not been declared.
This conceptual study reports no newly collected empirical data. Its proposed procedures require the prospective validation described above.
References
- Babu, Md. Asaduzzaman; Urmi, Hafsa Akter; Tofayel, Abzal Hosen; Sabuj, Md. Mehedul Islam; Alam, Md Tamjid Ul (2025). The impact of influencer marketing in the tourism industry: A digital marketing perspective. PLOS One. 20(12), e0338423. doi:10.1371/journal.pone.0338423.
- Piseddu, Tommaso; Vanhuyse, Fedra (2026). Existing evidence on the impact of climate risk on real estate valuations: a systematic map. Environmental Evidence. 15(1), 8. doi:10.1186/s13750-026-00389-6.
- Rosca, Cosmina-Mihaela; Stancu, Adrian (2026). Machine Learning Applications with Sensors for Indoor Air Quality Research. Sensors. 26(9), 2909. doi:10.3390/s26092909.
- Zhao, Li; Zhou, Jinwei; Cao, Jinde; Zhu, Weina (2025). EMSA: Explainable multilingual sentiment analysis models providing sentiment analysis across multiple languages. PLOS One. 20(11), e0333508. doi:10.1371/journal.pone.0333508.
- Hollin, Ilene L.; Ball, Jennifer Gerard (2022). Does price disclosure in pharmaceutical advertising result in price transparency? Evidence from a randomized experiment. Exploratory Research in Clinical and Social Pharmacy. 8, 100180. doi:10.1016/j.rcsop.2022.100180.
- Singh, Nutan; Shanmugam, Kavitha (2026). Exploring predictive models to improve the accuracy of Housing Price Index forecasts in India’s real estate sector. PLOS One. 21(1), e0341026. doi:10.1371/journal.pone.0341026.