Computer Vision Benchmarks Under Construction-Site Occlusion: A Targeted Narrative Synthesis and Protocol for Occlusion-Aware Evaluation

1. Introduction

Construction-site computer vision is often evaluated as if the scene were a stable indoor scan or a generic outdoor point cloud. That assumption is poor for active sites. The site contains unfinished assemblies, temporary installations, stacked materials, machinery, workers, and irregular scan paths. Relevant components are therefore only partly visible, and the resulting point clouds combine geometric incompleteness with density variation and sensor noise. LiDAR-based construction segmentation research describes these conditions directly, noting severe occlusion, clutter, varying point density, and noise, and linking them to blurred boundaries, missing data, and structural ambiguity in semantic segmentation. [1]

A benchmark intended for this domain should therefore do more than rank networks by aggregate mean intersection-over-union. It should test whether the model maintains semantic consistency as visible evidence decreases, whether errors concentrate at structural junctions, and whether synthetic training data transfer across occlusion patterns found on real sites. These questions matter because construction segmentation is not an isolated pattern-recognition task. It is used to support BIM synchronization, digital-twin updating, progress monitoring, and site management decisions in dynamic environments where errors can propagate into downstream planning or safety functions. [1] [2]

This article proposes a narrow intervention: a benchmark design for construction-site occlusion. The article is not a report of a completed dataset release or model comparison. Its aim is conceptual and methodological. It synthesizes targeted evidence from a small set of directly relevant studies and then specifies an evaluation protocol with operational definitions, algorithmic decision rules, outcome measures, and a falsifiable validation plan.

2. Research question and evidence scope

The research question is: how should a construction-site computer vision benchmark be structured so that model performance under occlusion is measured explicitly rather than treated as an unobserved source of error? The scope is limited to benchmark design for semantic perception of construction sites, with emphasis on 3D point clouds because the strongest direct evidence concerns point-cloud semantic segmentation under occlusion and domain shift. [1]

The evidence base is a targeted narrative synthesis rather than a systematic review. The direct core source is the BIM-guided virtual-to-real segmentation study, which explicitly identifies labeling difficulty, structural occlusion, sensor noise, and synthetic-to-real adaptation as central issues in construction-site point-cloud segmentation. A second construction source is used to frame site-level digital-twin management tasks in which segmented point clouds identify machinery, materials, and site features for operational decisions. Two transferable sources are used more cautiously: one provides an example of explicit scene-state representation and explainable, evidence-constrained digital-twin assessment from vision; the other discusses dataset diversity, generalization, validation weaknesses, and metric inconsistency in construction-related computer vision applications. A fifth source is treated only as indirect support for the value of integrating visual outputs with structured BIM and project data in downstream management contexts. [1] [2] [3] [4] [5]

This scope excludes claims about all construction vision tasks, all sensing modalities, or all forms of occlusion. It also excludes any assertion that current studies fail because their authors neglected benchmarking. The narrower claim is that existing evidence justifies a benchmark whose unit of analysis is occlusion-conditioned semantic reliability on construction scenes.

3. Prior findings relevant to benchmark construction

The direct construction segmentation evidence establishes three benchmark requirements. First, the benchmark must represent occlusion and clutter as ordinary scene properties rather than outliers. The construction point-cloud study attributes annotation difficulty and segmentation ambiguity to structural occlusion, noise, varying density, and missing data. Second, reporting should include both scene-level and component-level metrics because elongated or junction-forming components such as beams and columns are harder to classify than planar components with denser and more regular observations. Third, synthetic-to-real transfer should be evaluated explicitly because BIM-derived synthetic point clouds can encode transferable component semantics, but residual BIM–reality discrepancy and unmodeled site elements remain sources of semantic ambiguity. [1]

The digital-twin site-management study supports a different requirement: benchmark classes and errors should reflect downstream site tasks. That study uses point-cloud semantic segmentation to identify machinery, materials, and other site features for safety-oriented site management. It also notes limitations in data acquisition coverage and accuracy when using a handheld scanner. These observations imply that benchmark design should not assume full scene coverage and should include classes for temporary and movable site objects that may occlude structural elements while remaining operationally important themselves. [2]

Transferable methodological guidance comes from the vision-based digital-twin inspection framework. That work is not about construction-site occlusion benchmarking, but it shows how geometric measurements, semantic labels, image observations, and domain rules can be unified in an explicit scene-state representation with follow-up verification. For benchmark design, the important lesson is not the specific inspection domain. It is the value of auditable state variables and tool-mediated checks when image semantics alone are insufficient for reliable judgment. An occlusion benchmark can adopt this principle by recording visibility state, obstruction source, and evidence sufficiency alongside class labels. [3]

Review-level evidence on construction-related automation adds a caution about benchmark comparability. The review reports limited dataset diversity, weak generalization, small datasets in many AI-driven studies, inconsistent use of precision, recall, and mean IoU, and limited integration into BIM-based workflows. These are benchmark gaps rather than direct occlusion findings, but they support the need for standardized metrics, explicit validation splits, and reporting that separates classification success from deployment usefulness. [4]

There is also a boundary condition in the evidence. Synthetic-to-real success has been shown within similar residential building contexts, and the site-management study uses a specific scanning setup with practical coverage constraints. The evidence therefore does not justify assuming that a single benchmark design will generalize unchanged across tunnels, bridges, earthworks, industrial plants, or highly mobile open-air scenes. The benchmark proposed here is intended for building-construction sites captured as 3D point clouds under partial visibility. [1] [2]

4. Conceptual model of construction-site occlusion

For benchmarking purposes, occlusion should be defined operationally rather than visually. Let a target object or component c occupy a 3D support region S(c). Let V be the set of surface points from S(c) that are visible from the realized sensor trajectory after accounting for line-of-sight blocking by other surfaces and temporary objects. Define the visibility ratio as VR(c) = |V| / |S(c)|. In plain terms, VR is the proportion of the component surface support that could be observed from the scan, not merely the proportion that was correctly segmented. Occlusion severity is then OS(c) = 1 - VR(c). This benchmark treats partial hiding by permanent structure, temporary site elements, or severe self-occlusion from scan viewpoint as the same top-level phenomenon, while preserving the source of obstruction as a separate label.

Because ground-truth support S(c) is not always directly measurable in real scenes, the benchmark uses two annotation modes and a fixed geometric estimator. In synthetic scenes derived from BIM or simulation, S(c) is taken from the known component surface mesh after removing surfaces that are outside the benchmarked scan extent. In real scenes, S(c) is estimated from one prespecified reference source selected by a default hierarchy: (1) verified multi-pass fused reconstruction; (2) BIM aligned to observed as-built extent and manually corrected where the built state departs visibly from design; (3) manual geometric reconstruction when neither of the first two is available. After source selection, the same estimator is used in every scene: convert the reference for component c into a watertight or patchwise surface representation, clip it to the benchmarked spatial envelope, and compute |S(c)| as total reference surface area. Then estimate |V| by ray-casting from the recorded sensor trajectory or, if trajectory is unavailable, from registered scan origins, marking reference surface patches as visible when at least one unobstructed line of sight reaches the patch and a scan return falls within a fixed tolerance band of that patch. The tolerance band, mesh resolution, and clipping rule must be preregistered once for the benchmark release and not changed by scene. If two admissible reference sources are available and disagree, the higher-ranked source governs the primary stratum assignment; the lower-ranked source is retained only for sensitivity analysis. The chosen source, registration method, and any manual corrections must be recorded for each scene package. This rule is needed because BIM–reality discrepancy and unmodeled site elements can create ambiguity at structural junctions. [1]

The benchmark distinguishes four obstruction sources: structural overlap, temporary materials, machinery and equipment, and sensing-path limitation. Structural overlap covers one designed or built element blocking another. Temporary materials include stacked items, formwork remnants, coverings, or installations not represented in the planned structural model. Machinery and equipment include cranes, lifts, carts, and tools. Sensing-path limitation refers to incompleteness created chiefly by the chosen path, distance, or angle rather than by a physical blocker. This distinction is justified because prior construction studies report both environmental clutter and scanner coverage limitations as practical causes of incomplete evidence. [1] [2]

A benchmark centered on occlusion also needs to state what counts as a fair target. Some objects are so weakly observed that semantic judgment is no longer a property of the model but of missing evidence. The protocol therefore defines an evidence sufficiency flag with an uncertainty margin. A component is classed as evaluable if VR(c) >= tau_min + delta and non-evaluable if VR(c) < tau_min - delta, where tau_min is fixed before testing and delta is a preregistered annotation-uncertainty half-band. Components with VR(c) in [tau_min - delta, tau_min + delta) are assigned a border-zone flag. Under the default operational rule, border-zone cases are excluded from strict class scoring and reported separately; a sensitivity analysis then reallocates all border-zone cases first to the evaluable set and then to the non-evaluable set to test ranking stability. A hypothetical starting specification is tau_min = 0.10 with delta = 0.02. These values are design choices for prospective testing, not empirically established thresholds.

5. Proposed benchmark structure

The proposed benchmark, termed the Construction Occlusion Benchmark, is organized around scene packages rather than isolated scans. Each package contains: a raw point cloud or sequence of partial scans; sensor metadata; an aligned reference representation; semantic labels for structural and non-structural classes; occlusion annotations; and a benchmark card stating scene type, stage of construction, dominant occluders, and annotation confidence. Structural classes should at minimum cover floors, slabs, walls, columns, beams, ceilings or soffits, openings, and stairs where present. Non-structural classes should include temporary materials, machinery or equipment, protective barriers, and an explicitly defined residual class for other site objects. The residual class must be retained because prior construction work shows that heterogeneous non-modeled objects are common and create confusion near dominant structural components. [1] [2]

Each class instance is partitioned into visibility strata. A simple three-stratum rule is suitable for initial deployment: low occlusion if OS(c) < 0.25, medium occlusion if 0.25 <= OS(c) < 0.60, and high occlusion if OS(c) >= 0.60. These cut points are hypothetical starting values. They are deliberately broad because annotation error in real scenes will often exceed the precision implied by narrower bins. The benchmark score must always report performance by stratum and not only as a pooled average.

To summarize scene difficulty, the benchmark defines a Construction Occlusion Benchmark Index for scene s as a bounded descriptive score rather than an unconstrained mixture of quantities: COBI(s) = 100 × [w1 O(s) + w2 T(s) + w3 D(s) + w4 N(s)], with w1 + w2 + w3 + w4 = 1 and each wk in [0,1]. The default rule is equal weighting, wk = 0.25 for all k; a sensitivity analysis may repeat split stratification under an occlusion-emphasized alternative with w1 = 0.40, w2 = 0.20, w3 = 0.20, and w4 = 0.20. Each component is computed by one fixed procedure. O(s) is the mean occlusion severity OS(c) over evaluable structural instances in the scene and therefore lies in [0,1]. T(s) is normalized occluder diversity: if pj is the proportion of occluding points or occluding reference patches assigned to temporary-material, machinery-equipment, protective-barrier, and residual movable-object classes, then T(s) = -Σj pj log pj / log J, with J the number of represented occluder classes. Set T(s)=0 when J is zero or one; evaluate the logarithmic expression only for J >= 2. Zero-proportion terms contribute zero by continuity. D(s) is normalized point-density variability, computed by partitioning the scene into a preregistered cubic voxel grid, counting points per occupied voxel, calculating the coefficient of variation cv over occupied voxels, truncating cv at a preregistered cap cmax, and setting D(s)=min(cv,cmax)/cmax. N(s) is a normalized noise indicator derived from local geometric roughness: for each occupied voxel with enough neighbors, fit a local plane to its k-nearest points, compute the median absolute point-to-plane residual rv, truncate rv at a preregistered cap rmax, and aggregate by the scene median to obtain N(s)=min(median(rv),rmax)/rmax. The voxel size, neighbor count k, minimum-neighbor rule, and caps cmax and rmax must be fixed in the benchmark specification before any model comparison. COBI remains a scene-characterization variable for split balancing and error interpretation, not a performance metric. Require finite cmax > 0 and rmax > 0 before scoring; cmax is dimensionless, while rmax uses the same length unit as the plane residuals. Convert geometric coordinates to the registered common length unit before voxelization. A scene with no evaluable structural instances or no eligible local-plane fits receives a missing COBI value, rather than a zero, and is reported in a separate non-evaluable stratum. The benchmark must publish those counts.

The benchmark should include both synthetic and real subsets. The synthetic subset is needed because BIM-guided virtual scanning can generate large labeled point clouds and allows exact visibility computation. The real subset is required because the domain gap includes differences in noise, density, occlusion, reflection changes, and structural variation, and because unmodeled site elements are common in active construction. A benchmark that uses only one subset would miss either label control or real deployment conditions. [1]

6. Evaluation methodology and decision sequence

The benchmark protocol evaluates a model through a fixed decision sequence. Step 1 is preprocessing normalization. Raw input is voxelized or downsampled according to the model’s declared requirements, but the benchmark stores the original data and records the preprocessing operator. Step 2 is instance visibility assignment. For each reference instance c, compute or estimate VR(c) and assign an occlusion stratum and obstruction source. Step 3 is prediction alignment. Model outputs are mapped to benchmark labels, with any unmatched label mapped to the residual class. Step 4 is evaluability filtering. Strict class scoring uses evaluable instances only, while non-evaluable instances enter a separate uncertainty analysis. Step 5 is metric computation by class, scene, and occlusion stratum. Step 6 is robustness auditing, which compares degradation curves across strata and across synthetic-to-real splits.

The primary comparator family should include at least three model conditions rather than three named architectures. Condition A is a real-only supervised model trained on available real annotations. Condition B is a synthetic-only model trained on BIM-derived or simulated data without target-scene fine-tuning. Condition C is a synthetic-to-real adaptation model using explicit domain adaptation. This comparator logic follows the strongest direct source, which contrasts classical baselines with a BIM-guided fusion model trained on synthetic data and equipped to mitigate differences in noise, density, occlusion, and structural variation between synthetic and real scans. [1]

Primary outcome measures are stratum-specific mIoU, class-wise IoU for structural components, macro recall over temporary occluder classes, and a fitted occlusion trend coefficient. For each metric M, estimate the coefficient beta_M from a preregistered regression of the scene- or instance-level outcome on ordered occlusion stratum, using the same model form for all compared methods within a split. With three ordered strata, the default rule is a simple linear fit to the stratum means coded 0, 1, and 2; a more negative beta_M indicates faster performance decline as occlusion worsens. If an implementation chooses not to fit a trend model, it must report the simpler mean adjacent-stratum difference and label it as a step difference rather than a slope. Secondary measures are Cohen’s kappa, scene-level consistency, and boundary-error concentration at junction zones. Junction zones are reference neighborhoods around beam-column, wall-slab, wall-opening, and stair-landing interfaces, because prior work identifies structural ambiguity and blurred boundaries at such intersections. [1]

The protocol also evaluates confidence behavior on weak evidence, but it does so with an explicit overconfidence measure rather than calling every high-confidence prediction a calibration estimate. For instances with VR(c) < tau_min, the benchmark records a low-visibility overconfidence event when the model issues a class prediction with confidence q(c) >= tau_conf instead of abstaining. The default operational rule is tau_conf = 0.7, fixed before testing; sensitivity analyses may repeat reporting at 0.6 and 0.8. The primary summary is the Low-Visibility Overconfidence Rate, defined as the proportion of non-evaluable instances that trigger such an event. If a model does not expose confidence scores, it is assessed only on abstention frequency and strict class metrics. Standard empirical calibration measures may be added in future implementations, but this protocol makes the narrower claim that overconfident prediction under severe occlusion should be reported explicitly.

7. Validation design

Because this article does not report completed experiments, validation is specified prospectively. The benchmark should be tested through four nested tasks. Task 1 is annotation consistency: independent annotators assign visibility strata and obstruction sources on a subset of scenes, and agreement is measured separately from class-label agreement. Task 2 is threshold and reference sensitivity: benchmark outputs are recalculated after prespecified perturbations to tau_min, stratum cut points, and reference-source selection to determine whether scene labels or model rankings change materially. Task 3 is domain-transfer evaluation: the same model family is trained under the three comparator conditions and assessed on held-out real scenes stratified by COBI. Task 4 is downstream relevance: segmentation outputs are passed to a simple digital-twin site query layer that tests whether specified structural and temporary classes can be localized for management use. No task is claimed as completed here; the paragraph defines a falsifiable validation plan only. [2] [3]

The split logic should avoid leakage by grouping scenes at the construction-floor or site-package level rather than by random blocks of points. This is important because local fragments from the same site share geometry, scan trajectory, and occlusion structure. For synthetic-to-real evaluation, synthetic training scenes and real test scenes should be paired only at the level of broad building type or construction stage, not at the level of the same physical room reconstructed twice. The goal is to test transfer, not re-identification. [1]

A benchmark report should contain at least five tables or their functional equivalents: dataset composition by class and stratum; scene difficulty by COBI; model outcomes by comparator condition; exclusion, abstention, and low-visibility overconfidence outcomes for non-evaluable cases; and failure cases grouped by obstruction source. Exclusion must be reported as a benchmark property, not as a model-specific filtering choice: the number and proportion of instances with VR(c) < tau_min are shown for each test split and for each class and stratum transition. Strict class metrics are then computed on the same evaluable reference subset for all compared models within a split. If visual examples are provided, they should include the reference visibility mask, not only the predicted semantics. This makes it possible to inspect whether an apparent misclassification was instead a consequence of negligible visual evidence.

8. Comparison with specific prior methods

The proposed benchmark differs from the BIM-guided virtual-to-real segmentation framework in purpose. That framework is a segmentation method and training pipeline, with virtual LiDAR scanning, sliding-window preprocessing, feature fusion, point augmentation, and domain adaptation. The present proposal does not prescribe a network. It prescribes how such networks should be judged under occlusion. It extends the evaluation focus from overall transfer performance to visibility-conditioned reliability, obstruction-source effects, and confidence behavior on weak evidence. It is therefore complementary to, not a substitute for, model design. [1]

Relative to the digital-twin site-management application based on RandLA-Net, the benchmark proposal introduces two additions. First, it turns scanner limitations and partial coverage into annotated benchmark variables rather than uncontrolled acquisition artifacts. Second, it requires separate evaluation of structural classes and temporary operational classes, reflecting the fact that site-management decisions depend on both. The underlying application study shows why this matters: site personnel need location information about materials, machinery, and other features, but the study also acknowledges limits in coverage, efficiency, and accuracy of handheld data collection. [2]

Compared with the explainable indoor digital-twin assessment framework, the present proposal borrows the principle of explicit state representation but narrows it to benchmark variables that are measurable in construction scans: visibility ratio, obstruction source, evaluability, and junction-zone membership. It does not import the entire agent-based reasoning structure because benchmarking semantic perception does not require a full interactive inspection agent. The transferable point is auditability: predictions should be interpreted against recorded evidence states rather than only against final class labels. [3]

The proposal also responds to benchmark weaknesses noted in the broader construction automation review. That review points to limited dataset diversity, incomplete validation, and inconsistent metrics. The response here is concrete: scene-level grouping to reduce leakage, mandatory reporting by occlusion stratum, and a distinction between scene characterization variables and performance metrics. These additions are not claimed to solve generalization, but they make it easier to detect when a model performs well only on low-occlusion subsets or on narrow scene types. [4]

9. Failure cases, conflict points, and limits of generalization

Several failure cases are predictable. The first is occlusion-label uncertainty in real scenes where the reference model is itself incomplete. If S(c) is approximated poorly, VR(c) becomes unstable and stratum assignment may be wrong. The second is residual-class inflation. Prior work notes that broad Other categories contain heterogeneous objects with ambiguous geometry; the same issue can obscure whether the model failed due to occlusion or due to class heterogeneity. The third is self-occlusion of elongated components such as beams and columns, whose narrow visible support can produce unstable IoU estimates. The fourth is scanner-path confounding: a scene may appear difficult because of poor acquisition rather than genuine construction clutter. [1] [2]

There is also a substantive conflict in benchmark philosophy. One can argue that a benchmark should score only visible evidence and ignore the unseen portion, because segmentation is not expected to infer hidden geometry. The alternative view, adopted here, is that the benchmark should record hidden extent explicitly because practical reliability depends on how performance deteriorates as evidence disappears. These positions are not logically inconsistent; they answer different questions. The proposed protocol resolves the tension by reporting both visible-support class performance and evaluability-aware calibration outcomes.

Generalization is intentionally limited. The protocol is designed for construction-site point-cloud semantics in building contexts where BIM or reference geometry can be aligned with scans. It is not automatically suitable for monocular image benchmarks, for moving-object tracking, or for project-level risk prediction. The source on BIM-integrated predictive analytics is relevant only in a downstream sense: structured project data may eventually consume benchmarked perception outputs, but that does not validate the perception benchmark itself. [5]

10. Conclusion

This article established a bounded conceptual contribution: a benchmark design and evaluation protocol for construction-site semantic perception under occlusion. The targeted evidence shows that construction point clouds are routinely affected by occlusion, clutter, density variation, noise, and BIM–reality discrepancy; that segmented point clouds support digital-twin and site-management tasks; and that explainable vision systems benefit from explicit state variables and auditable evidence structures. From this basis, the article proposed operational definitions for visibility ratio, occlusion severity, evaluability, obstruction source, and a scene-level difficulty index; specified comparator conditions for real-only, synthetic-only, and synthetic-to-real models; and defined outcome measures that expose degradation under worsening visibility rather than hiding it within aggregate scores. [1] [2] [3] [4]

What remains untested is substantial. No empirical benchmark has yet been assembled under the full protocol, no thresholds have been optimized for annotation reliability, and no evidence currently shows which metric combination will be most stable across sites. The proposed cut points and confidence rules are therefore hypothetical design choices to be evaluated prospectively. The next step is implementation and falsification: constructing scene packages, measuring annotation agreement, and determining whether occlusion-aware reporting changes model ranking or improves the interpretability of synthetic-to-real transfer in active construction environments.

References

1. Zou, Yiquan; Liang, Tianxiang; Nabi, Jafri Syed Riaz un; Xu, Zhendong; Zhou, Liang; Xiong, Biao (2026). A BIM-Guided Virtual-to-Real Framework for Component-Level Semantic Segmentation of Construction Site Point Clouds. Sensors. 26(1), 308. doi:10.3390/s26010308.
2. Qin, Wenli (2026). Application of construction site management based on digital twin and point cloud semantic segmentation technology. PLOS One. 21(3), e0340274. doi:10.1371/journal.pone.0340274.
3. Jing, Zijian; Zhu, Liyi; Chen, Tianyi; Hovestadt, Ludger; Li, Li (2026). Vision-Based Digital Twin and AI Agent Framework for Low-Cost, Explainable Indoor Building Inspection and Safety Assessment. Sensors. 26(15), 4992. doi:10.3390/s26154992.
4. Haq, Moinul; Mohsen, Osama; Elnabwy, Mohamed T.; Martinez, Pablo (2026). Integrating automation into construction waste quantification: A review of digital innovations for sustainable resource management. Waste Management &amp; Research: The Journal for a Sustainable Circular Economy. 44(9), 1308-1326. doi:10.1177/0734242x261441094.
5. Li, Ruixin; Almenweer, Reem A.; Isleem, Haytham F.; Metwally, Ahmed Sayed M. (2026). Optimizing construction risk management through BIM integration and strategic planning. Scientific Reports. 16(1), 28592. doi:10.1038/s41598-026-64364-4.