Computer Vision for Property Inspection with Abstention: A Digital-Twin and Conformal Prediction Framework for Defect Detection and Scene-Level Safety Assessment

1. Introduction

Visual property inspection includes two related task families. The first is localized defect or hazard detection, such as identifying cracks, spalling, obstructed circulation, missing fixtures, or unstable stored items. The second is scene-level safety assessment, such as determining whether a room, stair, corridor, exterior area, or work zone satisfies predefined safety propositions that depend on multiple observations and measurements. Images, video, UAV imagery, and point clouds make both task families increasingly tractable, but a system that always issues a definitive judgment is poorly matched to inspection practice. Occlusion, blur, poor illumination, reflective surfaces, incomplete coverage, and domain shift often make visual evidence insufficient for a safe conclusion.

A direct precedent comes from indoor residential safety assessment. One reported framework converts continuous smartphone video into a metric-scale point cloud, binds geometry to semantic entities and image observations, formalizes risk guidelines into computable structures, and supports follow-up verification through targeted measurement and visual cross-checking. This is relevant because it shows that visual inspection can produce a structured factual record rather than only flat labels, and that inspection can proceed iteratively when initial evidence is incomplete. [1]

Related inspection domains reinforce both feasibility and caution. Remote bridge inspection literature describes workflows for three-dimensional reconstruction, damage identification, and non-contact measurement within digitalized inspection systems. Construction-site safety research combines point-cloud acquisition, semantic segmentation, and digital-twin concepts to identify machinery, materials, and risk factors in dynamic environments. These studies show that visual evidence can be organized at asset scale, but they also indicate that digital methods must fit established inspection practice and remain at least as reliable as human inspection before operational adoption. [2] [3]

This article addresses a narrower question than general automated inspection: how should a property inspection system represent uncertainty and abstain explicitly when available evidence does not support a reliable defect or safety judgment? The contribution is a technically explicit conceptual framework that combines digital-twin scene integration, conformal class-label prediction sets, measurement-aware abstention, and staged escalation for visible property inspection tasks where false certainty is unacceptable. [4]

2. Research question and evidence scope

The research question is: how can a computer-vision workflow for property inspection produce actionable and auditable outputs while withholding judgments that are unsupported by available evidence? The domain is restricted to visible conditions observable from images, video, or point clouds. Example targets include crack-like defects on structural or finish surfaces, missing or misplaced fixtures, clutter in circulation areas, inadequate clearances, unsafe thresholds, and site-level hazards detectable from object identity and spatial arrangement. Hidden defects requiring destructive testing or subsurface sensing are outside scope.

The evidence scope is a targeted narrative synthesis of direct application and transferable methods. Indoor building safety assessment provides evidence that monocular video, metric reconstruction, semantic parsing, and computable risk evaluation can be linked in one workflow. Remote bridge inspection shows how reconstruction, damage detection, and digital asset representations are integrated in practice. Bridge crack inspection explains why some defect tasks require pixel-level characterization rather than object boxes alone. Construction-site safety work shows how point-cloud semantic segmentation supports risk identification in changing environments. Digital-twin architecture work contributes a layered design for multi-source fusion and decision support. Inductive conformal prediction contributes a formal approach to class-label uncertainty and abstention in object detection. [1] [2] [5] [3] [6] [4]

This manuscript is not a systematic review and does not claim exhaustive coverage. Its aim is narrower: to assemble sufficient direct and transferable evidence for an abstention-aware inspection framework. The argument therefore distinguishes among findings established by prior studies, design elements transferred into the property-inspection problem, and propositions that remain untested until future empirical evaluation.

3. Prior work relevant to inspection with abstention

The indoor safety framework is important because it goes beyond generic recognition. It reconstructs a metric point cloud from smartphone video, uses fiducials to set absolute scale, parses geometry into semantic entities and relationships, and stores evidence in a hierarchical scene representation. Risk knowledge is formalized as computable prototypes, and an agent can execute an initial scan followed by targeted verification when the existing scene state is insufficient. For property inspection, two lessons are transferable: visual evidence can be represented as an auditable state linking geometry, semantics, and images, and unresolved questions can trigger additional measurements rather than a forced judgment. [1]

The bridge inspection review frames remote inspection as visual assessment of visible surface conditions using camera sensors and LiDAR, in contrast to structural health monitoring based on continuous sensor measurements. It identifies three-dimensional reconstruction, automatic damage identification, and non-contact physical measurement as major parts of current digitalized inspection. It also states a practical boundary condition: computer-vision-based inspection should be at least as reliable as human inspection when integrated into existing frameworks. For the present topic, that requirement supports abstention. When evidence is weak, a safety-related system should preserve reliability by declining unsupported conclusions. [2]

Bridge crack research contributes a more specific lesson. The cited study explains that object-level crack detection is often insufficient because crack integrity depends on shape, width, length, and spatial distribution, and therefore motivates semantic segmentation for pixel-level characterization. The same discussion notes that some approaches remain sensitive to complex backgrounds and noise. For property inspection, this implies that certain defect classes require segmentation or contour extraction, and that visually ambiguous backgrounds are expected inspection conditions that should sometimes trigger abstention rather than forced classification. [5]

Construction-site safety work contributes the argument that static BIM models alone are limited in dynamic environments containing changing objects, machinery, and materials. That study presents digital twins as a way to process large volumes of dynamic data for monitoring and decision support, and it emphasizes that point clouds become operationally useful only after semantic interpretation. This transfers directly to property inspection tasks such as egress obstruction or clearance violation, where risk depends on both what objects are present and where they are located relative to circulation or functional zones. [3]

Inductive conformal prediction addresses uncertainty more directly. The cited object-detection study states that standard inductive conformal prediction yields finite-sample marginal coverage under the assumption that calibration and test detections are exchangeable in the i.i.d. sense. It also distinguishes marginal from class-conditional coverage and shows that the former does not guarantee the latter. In addition, it treats adaptivity as practically important: hard instances should receive larger prediction sets than easy ones. For inspection, these properties are useful because they convert uncertain detections from opaque score behavior into explicit prediction sets that can support abstention and escalation. [4]

4. Conceptual framework and operational definitions

The proposed framework treats property inspection as evidence-constrained scene interpretation. Its unit of analysis is an inspection target T, which may be a localized candidate region, a building element, or an entire scene. Each target is associated with a record R(T) containing image evidence, geometric evidence, semantic predictions, derived measurements, uncertainty metadata, and the reasons for any final decision. The framework permits three action states at each decision point: positive finding, negative finding, and abstain. A positive finding means the evidence supports the presence of a predefined defect or risk proposition. A negative finding means the evidence supports its absence relative to the operational rule. Abstain means neither conclusion is supported by the available evidence.

A localized defect is defined as a visible condition attached to a component or image region and assigned a label from a finite foreground set L_f, for example crack, spall, obstruction, missing grab bar, or unstable material stack. Background is treated separately as a non-foreground label b. A scene-level risk proposition p is a Boolean statement over one or more entities and measurements, such as 'circulation width in the path segment is below the threshold' or 'support fixture required for transfer is absent in the designated zone'. A scene-level safety assessment consists of evaluating a finite proposition set P for a defined spatial unit.

Rule 1 governs conformal class-label outputs for each detection candidate x. Let the admissible label set be L = L_f union {b}. A conformalized detector returns C(x) subset of L, where C(x) = {y in L : s(x,y) <= q_hat_alpha}. Here s is a nonconformity score and q_hat_alpha is the empirical calibration quantile at significance level alpha. The implementation distinguishes four cases. If C(x) = {b}, the candidate is rejected as background. If C(x) contains exactly one foreground label and not b, the candidate is accepted as that defect or hazard. If C(x) contains multiple labels, whether foreground-foreground or foreground-background, the system abstains because class membership is unresolved. If C(x) is empty, the system also abstains; empty set and background singleton are operationally distinct because the former is a threshold-based rejection of all labels, whereas the latter is a confident non-foreground decision. [4]

Rule 2 governs measurement sufficiency. For every quantitative proposition p, define its orientation o(p) in advance. If risk increases when a measurement is less than a threshold tau, then o(p) = low-is-risky and the proposition is m < tau. If risk increases when a measurement exceeds tau, then o(p) = high-is-risky and the proposition is m > tau. Let the estimated measurement interval be [l,u]. If o(p) = low-is-risky, decide present only when u < tau, decide absent only when l >= tau, and otherwise abstain. If o(p) = high-is-risky, decide present only when l > tau, decide absent only when u <= tau, and otherwise abstain. Any overlap between the uncertainty interval and the decision boundary yields abstention.

Scene-level aggregation uses tri-state proposition logic. Each proposition in P takes one of three values: present, absent, or unresolved. The scene is classified as safe only if every proposition is absent. The scene is classified as unsafe if at least one proposition is present, even if other propositions are unresolved. The scene is classified as inspection inconclusive only when no proposition is present and at least one proposition is unresolved. This makes the mixed case explicit: one confirmed hazard is sufficient to reject a safe classification, but unresolved propositions remain visible in the report.

5. Proposed system architecture and algorithm

The architecture has four layers: acquisition, scene construction, inference, and decision governance. This separation is informed by digital-twin work that distinguishes system construction, data acquisition, and higher-level generation or decision functions, and by bridge digital-twin descriptions that separate acquisition from analytics and visualization. The present adaptation treats abstention as an explicit design objective rather than an afterthought. [6] [2]

The acquisition layer accepts one or more visual streams according to inspection context: smartphone video for interiors, still photographs for close defect documentation, handheld or mobile LiDAR when available, and UAV imagery for elevated or difficult-to-access surfaces. Inputs must be time-stamped and either directly spatially registered or retrospectively alignable. Bridge and construction studies support the practical relevance of UAV imagery, camera sensing, LiDAR, and SLAM-derived point clouds in remote or complex inspection settings. [2] [3]

The scene construction layer produces a structured property twin. A minimal instance contains a metric point cloud or mesh, camera poses and keyframes, semantic entities, links from entities to source images, and storage for derived measurements. The indoor safety framework provides a close precedent through its hierarchical scene representation linking geometric measurements, semantic labels, image observations, and regulatory knowledge. The present proposal extends that idea by requiring uncertainty fields for every stored entity and measurement so that abstention can be computed and audited directly. [1]

The inference layer contains three modules. Module A performs candidate detection and must expose a complete per-class score vector over L rather than only a top-1 label, because conformalization depends on access to all relevant class scores. Module B performs pixel-level segmentation or contour extraction for defect classes whose morphology is inspection-relevant, such as cracks. Module C computes geometric and relational quantities from the property twin, including widths, clearances, proximities, support presence, and field-of-view coverage. The indoor safety source supports targeted measurement from selected keyframes lifted into metric three-dimensional space, while the crack literature supports pixel-level analysis when morphology matters. [4] [1] [5]

The algorithm proceeds in eight stages: define the inspection scope S, including spatial units, labels, propositions, and threshold orientations; acquire initial imagery and any available point clouds; reconstruct and align the scene into the property twin; run candidate detectors, segmentation modules, and geometric extractors; conformalize each candidate using a calibration set drawn from the intended deployment domain; evaluate measurements with Rule 2; aggregate proposition states into target-level and scene-level outcomes; and resolve unresolved outputs by follow-up acquisition or referral to a human inspector. Follow-up acquisition is central rather than optional. If a doorway-width interval straddles a threshold, the system may request a closer view. If a crack candidate receives a conformal set containing both crack and background, the system may request a higher-resolution still image. If a proposition depends on information not visually decidable, such as whether an item is permanently fixed, the system escalates directly to human review. [1]

6. Variables, assumptions, comparators, and outcome measures

The main design variables are modality configuration M, base detector architecture D, segmentation requirement G for each defect class, nonconformity function N, significance level alpha, proposition library P, and escalation policy E. M may distinguish video-only, video plus LiDAR, still-image-only, or hybrid configurations. G indicates whether a defect class requires only candidate localization or additional mask-based morphology. E specifies whether unresolved cases trigger automatic follow-up acquisition, immediate human review, or both.

Dependent variables include candidate-level decision state, proposition-level decision state, scene-level decision state, number of follow-up requests, human-review referral rate, closure rate after follow-up, and elapsed time to case closure. Derived uncertainty variables include prediction-set size |C(x)|, empty-set rate, rate of background singletons, measurement-interval width, and entropy over normalized class-score vectors when available as a descriptive proxy for difficulty. Because the conformal study distinguishes marginal from per-class behavior, both aggregate and class-specific summaries are needed. [4]

The framework rests on several assumptions. The relevant condition must be visually observable. Calibration data must be sufficiently exchangeable with deployment data for the marginal coverage interpretation of inductive conformal prediction to remain meaningful. Spatial registration must be accurate enough for any measurement-based proposition used in decision-making. Proposition definitions and threshold orientations must be specified before inference. At least some unresolved visual cases must be improvable through better views or higher resolution. Human inspection must remain available as the escalation path for ambiguities that vision or registration cannot resolve. [4] [1]

A future empirical study should include at least two comparators. The first is a forced-decision baseline that assigns top-1 labels to all candidates and binary threshold decisions to all propositions. The second is confidence-threshold rejection without conformalization. These comparators would show what abstention changes, and whether conformalized prediction sets provide a better basis for abstention than heuristic score cutoffs. This design is consistent with the bridge literature's requirement that new digital procedures be reliable enough for integration into established inspection frameworks. [2] [4]

Primary outcome measures should include marginal coverage of conformal label sets, class-specific coverage reported separately, average prediction-set size, candidate-level abstention rate, proposition-level abstention rate, scene-level inconclusive rate, and correctness among decided cases. For defect tasks, correctness may be summarized using sensitivity and positive predictive value against adjudicated references. For scene-level judgments, agreement with expert scene conclusions and evidence-trace completeness are needed. Auditability should be operationalized as the proportion of outputs for which the report contains retrievable image evidence, geometric evidence where applicable, and the specific rule used to reach the outcome. [4] [1]

7. Prospective evaluation protocol

Because no new experiment is reported here, this manuscript replaces an empirical results section with a prospective evaluation protocol. The target population would consist of property-inspection cases requiring visible defect detection or scene-level safety assessment. Sampling should cover at least three scenario families: indoor residential or commercial spaces, exterior building elements accessible by camera, and site-like areas containing movable hazards. Bridge imagery may be used as a transfer stress test for defect morphology and difficult viewing conditions, but it should not be treated as identical to the property domain without separate validation.

Reference annotation should be produced by domain experts using adjudication that assigns each target proposition one of three labels: present, absent, or indeterminate due to insufficient evidence. The same tri-state convention should be used for scene-level conclusions. This is necessary because the purpose of the framework is to evaluate abstention rather than force a binary answer when even the reference evidence is inconclusive. For localized defects, reference annotation should include class labels and masks or contours when morphology matters. For proposition-based risks, the record should identify the exact proposition, any threshold involved, and the evidence used by the experts.

The scoring convention must handle indeterminate references explicitly. If expert adjudication for a target or scene is indeterminate, that case is counted as a human abstention reference and is excluded from binary correctness metrics that presuppose a resolved truth state. It is not counted as an error for an automated abstention. Instead, two additional measures are reported: abstention concordance, defined as the proportion of human-indeterminate cases on which the system also abstains, and unsafe-overcall rate on indeterminate references, defined as the proportion of human-indeterminate cases on which the system issues a positive defect-present or unsafe conclusion. By contrast, if the human reference is determinate and the system abstains, the abstention is not scored as correct or incorrect in binary accuracy metrics; it contributes to abstention rate and should be analyzed through selective reporting on the decided subset.

The main comparative study should evaluate four systems: a forced-decision top-1 baseline, the same base models with simple confidence-threshold rejection, the proposed conformal-abstention framework without follow-up acquisition, and the full framework with follow-up acquisition plus measurement-based abstention. This isolates three effects: the value of abstention itself, the added value of conformalized label sets beyond heuristic score rejection, and the value of iterative evidence gathering. Results should be reported separately for candidate-level defects, proposition-level risk rules, and complete scene inspections. [4] [1]

A critical stress-test suite should include low illumination, reflective surfaces, repetitive textures, clutter, severe occlusion, hairline defects, motion blur, and contradictory evidence across views. The crack literature identifies complex backgrounds as a recurring source of error, while the bridge review emphasizes trade-offs in accuracy, coverage, and cost across sensing methods. Under these conditions, an abstention-aware system should increase unresolved outputs rather than maintain constant decision rates. A future empirical study should therefore test whether increased abstention under difficult conditions is associated with reduced false certainty and more appropriate escalation to human review. [5] [2]

8. Comparison with prior methods, boundary conditions, and conclusion

The proposal extends rather than replaces the indoor safety assessment approach. That earlier framework already provides a structured scene state, formalized risk prototypes, and a dual-pathway process of initial assessment plus follow-up verification. The present manuscript adds calibrated abstention as a first-class output at three levels: candidate classification, measurement-based proposition evaluation, and scene conclusion. The bounded contribution is therefore not a general digital-twin architecture, but the explicit integration of abstention logic into a scene-based property inspection workflow. [1]

Relative to remote bridge inspection methods, the present framework is intentionally narrower. Bridge reviews encompass reconstruction, defect detection, structural measurements, data handling, and management-system integration. The current proposal does not attempt to cover continuous structural monitoring or full asset-management functions. Its scope is limited to visible defect and safety judgments represented within a property twin and governed by abstention rules. This restriction matters because the bridge literature itself notes that emerging visual methods must fit existing inspection procedures and satisfy reliability expectations before adoption. [2]

Relative to bridge crack segmentation models, the proposal is deliberately model-agnostic. The cited crack study reports why segmentation is useful and describes a lightweight network for UAV-based bridge inspection, but those findings do not establish that the same architecture is suitable for property defects such as moisture staining, missing fixtures, or occupancy-related hazards. The present framework therefore treats segmentation as a class-dependent requirement rather than endorsing one network across all property inspection tasks. [5]

Relative to construction-site digital-twin and point-cloud work, the proposal broadens safety assessment beyond extreme-weather site preparation to property-inspection propositions in general. It preserves the insight that dynamic environments require semantic interpretation of point clouds and that digital twins are useful where static models alone cannot represent current conditions. Transfer is plausible for environments with changing occupancy or movable hazards, but generalization weakens when reliable registration cannot be maintained or when the domain lacks stable semantic categories for proposition-based reasoning. [3]

The strongest boundary condition concerns uncertainty guarantees. Standard inductive conformal prediction offers marginal coverage under exchangeability, not per-class guarantees. In safety inspection, rare classes may be the most consequential. Severe class imbalance or domain shift could therefore yield acceptable marginal behavior while masking inadequate coverage for specific hazard categories. The conformal study notes that class-conditional guarantees require different procedures such as Mondrian calibration. For property inspection, class-specific calibration is a design option to be tested, not an established solution already validated by the evidence reviewed here. [4]

Several failure modes are foreseeable. Hidden defects and subsurface deterioration are beyond the reach of the visual evidence considered here. Reconstruction or scale errors can propagate into misleading measurements and mistaken proposition states. Semantic ambiguity may be irreducible from vision alone, for example when temporary clutter and permanent unsafe layout appear similar in available images. Calibration shift across devices, property types, lighting conditions, or rare hazard classes can invalidate expected conformal behavior. Proposition libraries can also be incomplete; abstention can represent insufficient evidence, but it cannot compensate for omitted domain rules. [1] [4]

The literature reviewed establishes several points conceptually. Vision-based and point-cloud workflows can support indoor safety assessment, bridge defect inspection, and construction-site risk identification, and digital-twin representations can organize heterogeneous geometric and visual evidence for later analysis and querying. Conformal prediction can convert detector scores into prediction sets with finite-sample marginal coverage under stated assumptions. Building on those findings, this article has proposed a bounded framework for visual property inspection in which automated outputs may be positive, negative, or explicitly inconclusive. The framework specifies candidate-level and measurement-level abstention rules, a scene aggregation rule, a layered architecture, an inspection algorithm, a scoring convention for indeterminate human references, and a prospective validation design. What remains untested is substantial: whether the proposed abstention logic improves reliability in property inspection, whether follow-up acquisition closes enough unresolved cases to justify its cost, which nonconformity functions are best suited to property defects, and how often scene-level outputs remain inconclusive in realistic workflows. The manuscript therefore establishes a conceptual and methodological basis for abstention-aware property inspection, not an achieved performance claim. [1] [2] [3] [4]

References

1. Jing, Zijian; Zhu, Liyi; Chen, Tianyi; Hovestadt, Ludger; Li, Li (2026). Vision-Based Digital Twin and AI Agent Framework for Low-Cost, Explainable Indoor Building Inspection and Safety Assessment. Sensors. 26(15), 4992. doi:10.3390/s26154992.
2. Ribeiro, Diogo; Rakoczy, Anna M.; Cabral, Rafael; Hoskere, Vedhus; Narazaki, Yasutaka; Santos, Ricardo; Tondo, Gledson; Gonzalez, Luis; et al. (2025). Methodologies for Remote Bridge Inspection—Review. Sensors. 25(18), 5708. doi:10.3390/s25185708.
3. Qin, Wenli (2026). Application of construction site management based on digital twin and point cloud semantic segmentation technology. PLOS One. 21(3), e0340274. doi:10.1371/journal.pone.0340274.
4. Mohammed, Mohammed Aliy; Anzaku, Esla Timothy; Jonkers, Jef; Krishnamoorthy, Janarthanan; Neve, Wesley De; Van Hoecke, Sofie (2026). Inductive Conformal Prediction for Guaranteed Class-Label Coverage in Object Detection. Journal of Imaging. 12(8), 348. doi:10.3390/jimaging12080348.
5. Dong, Xuwei; Yu, Zhibing; Yuan, Jiashuo; Dai, Jinpeng (2026). Automatic bridge crack detections: A lightweight segmentation model for UAV inspection. PLOS One. 21(9), e0358865. doi:10.1371/journal.pone.0358865.
6. Gong, Li; Ding, Menglong (2025). Urban traffic digital twin system development in unity. Scientific Reports. 15(1), 40085. doi:10.1038/s41598-025-23943-7.