Abstract
Construction progress tracking is often evaluated through isolated models and site-specific datasets, which makes it difficult to compare methods that use BIM, point clouds, or digital twins. This manuscript proposes a reproducible benchmark design for three related tasks: progress detection, site state inference, and sequential progress prediction. The proposal is grounded in a targeted narrative synthesis of prior work that has used BIM as a structured construction information source, synthetic point clouds generated from BIM, digital-twin site representations, and probabilistic or sequential models for time-dependent construction inference. The benchmark design is intended to unify these strands without assuming that any one sensing modality is sufficient in all settings. It specifies a reproducible pipeline for dataset generation, a standardized state schema, labeling rules, comparator classes, and an evaluation protocol that can be executed on both synthetic and real site data. A central design choice is to separate observed geometry from inferred work state: point clouds and BIM-derived attributes provide evidence, while progress labels are generated through explicit operational rules rather than implicit manual judgments. The manuscript also defines boundary conditions, including occlusion, semantic class mismatch, incomplete BIM content, and uncertainty in temporal alignment. The result is a benchmark protocol that is precise enough for replication and flexible enough to support controlled transfer across residential, industrial, or mixed-site contexts, while leaving empirical validation for future execution.
1. Research question and evidence scope
The research question is how to define a benchmark for construction progress tracking that remains reproducible across data sources and model families while still reflecting the operational constraints of BIM-based, point-cloud-based, and digital-twin-based workflows. The present manuscript is a conceptual research article. It does not report a new experiment. Instead, it synthesizes a targeted body of prior work to define a benchmark architecture that can support three tasks: detecting whether a work package has advanced, inferring the current site state from multi-modal inputs, and predicting the next progress state in a sequence. The scope is deliberately narrow. Evidence is drawn from studies that directly inform benchmark construction: BIM-integrated predictive frameworks, synthetic point cloud generation from BIM, digital-twin site management with point cloud segmentation, belief-based causal modeling of AI-enabled BIM digital twins, and reproducible simulation pipelines for generating standardized datasets. Studies on broader BIM adoption, sustainability, materials selection, or generalized automation agendas are excluded unless they clarify benchmark design decisions. [1] [2] [3] [4] [5]
The core methodological problem is not whether progress tracking is useful, but how to create a shared evaluation substrate. Existing work shows that BIM can supply structured project features for prediction, that synthetic point clouds can reduce annotation cost, and that digital twins can connect observed site conditions to decision support. At the same time, these studies also expose fragmentation: some emphasize optimization, others segmentation, and others expert belief modeling. A benchmark must therefore define common input-output relations, a common temporal unit, and a common representation of progress state so that different methods can be compared without collapsing their distinct assumptions. [1] [2] [3] [4]
2. Prior methods and what they imply for benchmark design
Prior work suggests three technical lessons. First, BIM-based methods can support sequential inference when project attributes are transformed into structured features that encode schedule variation, resource use, cost deviation, and risk context. In HCIF-Net, TabNet, Bayesian networks, temporal LSTM, and Monte Carlo simulation were coordinated within one framework for risk management and sequential progress prediction. For benchmark design, the relevant lesson is not the reported performance but the need for a feature schema that separates static project descriptors from time-indexed evidence and from latent risk variables. [1]
Second, BIM-to-real point cloud transfer shows that annotation-free or low-annotation evaluation is feasible when synthetic point clouds are generated from BIM with component-level labels. The BIM-guided virtual-to-real framework demonstrates that a synthetic corpus can be created at scale, with controlled occlusion, density, and viewpoint variation, and then used to test whether segmentation predictions generalize to real scans. For a progress benchmark, this suggests that labels should be generated in a reproducible pipeline rather than inferred ad hoc from site photographs or manual inspection alone. It also indicates that the benchmark should preserve the distinction between component semantics and progress semantics, because these are not identical. [2]
Third, digital-twin site management based on point cloud semantic segmentation shows that site state inference can be connected to operational uses such as locating materials, machinery, buildings, and potential risk factors within a registered 3D site model. The reported approach is directed to safety management and layout control under changing site conditions, not to benchmarked progress-state sequences. For benchmark design, the supported implication is narrower: segmentation outputs and registered digital-twin representations can serve as evidence channels for site-state description, but a progress benchmark still needs a separate state taxonomy and explicit transition labels. [3]
Fourth, the Bayesian-network study of AI-enabled BIM digital twins demonstrates a useful distinction between empirical evidence and structured expert belief. The model treats progress-related outcomes as scenario-dependent probabilities derived from elicited judgments rather than observed performance. For a reproducible benchmark, this supports a design in which uncertain or missing labels can be represented explicitly rather than silently converted into hard ground truth. However, it also warns against mixing belief-based scenario analysis with benchmark scoring unless the label provenance is declared and separated from measured outcomes. [4]
Finally, the reproducible HVAC dataset paper contributes a transferable procedural principle: a benchmark can be designed as a deterministic generation pipeline with fixed seeds, pinned software versions, and a minimal schema shared across heterogeneous environments. Although the domain is building control rather than construction progress, the design logic is directly relevant. A benchmark for progress tracking should likewise constrain what variables are exported, define what is intentionally excluded, and document the pipeline so that data generation is reproducible across buildings and simulation backends. [5]
3. Benchmark tasks and operational definitions
The proposed benchmark contains three tasks. Task A, progress detection, asks whether a defined work package has advanced since a prior observation window. Task B, site state inference, asks which progress state best describes the site at time t from the available modalities. Task C, sequential progress prediction, asks for the next state or state trajectory given a history of observations. These tasks are related but not interchangeable: a method may detect change accurately while failing to infer state class or predict the next transition. [1] [3]
The benchmark uses a state vector s(t) composed of four groups of variables: geometric evidence G(t), semantic evidence M(t), schedule position P(t), and uncertainty flags U(t). Geometric evidence may include point-cloud density, registered component extents, or spatial occupancy differences. Semantic evidence may include BIM object class, as-built component presence, and labels derived from segmentation. Schedule position may include planned activity identifier, planned start and finish, and relation to baseline schedule. Uncertainty flags represent occlusion, incomplete capture, missing BIM content, or unresolved class ambiguity. The state vector is not a model output; it is the benchmark’s canonical representation of the site at a time point. [2] [3] [1]
Default operational rule: for each work-package family f, preregister a completion threshold theta_f = 0.80, a registration tolerance delta_f, and an admissible spatial-relation set R_f before any evaluation. For each required component c in work package w at time t, assign component status z_c(t) by the following rule: z_c(t) = present only if (i) the component is detected in the observation, (ii) its registration error to the BIM reference is less than or equal to delta_f, and (iii) its observed relation to neighboring required components and host elements belongs to R_f; z_c(t) = absent if the capture is adequate and any of these conditions fails; z_c(t) = indeterminate if the local evidence is occluded, too sparse for registration, or missing because BIM-to-scan alignment failed. Let n_present(w,t) be the number of required components with z_c(t) = present, n_required(w) the number of required components in the package definition, and q_indet(w,t) the fraction with z_c(t) = indeterminate. Then assign the package label advanced only if n_present(w,t) / n_required(w) >= theta_f and q_indet(w,t) <= 0.20; assign not advanced if n_present(w,t) / n_required(w) < theta_f and q_indet(w,t) <= 0.20; otherwise assign uncertain and exclude that case from the primary ranking score while retaining it for sensitivity analysis. Default tolerances should be derived from the BIM object scale: set delta_f to the larger of a fixed scan-resolution bound and 5% of the component’s longest BIM dimension, and define R_f as a preregistered relation class such as attached-to, supported-by, enclosed-within, or aligned-with according to the package specification. Sensitivity analysis should vary theta_f over a fixed preregistered grid and report whether rankings change under those alternatives. [2] [3]
To avoid semantic confusion, the benchmark distinguishes component-level completeness from package-level progress. For example, walls, beams, floors, and slabs may be correctly segmented, yet the work package may remain incomplete if non-structural elements required by the package definition are absent. This distinction reflects the BIM-guided segmentation literature, where component recognition is accurate but the residual “other” category can hide omitted auxiliary elements. The benchmark must therefore store both recognized components and missing-required-components as separate outputs. [2]
4. Reproducible dataset generation pipeline
The benchmark should be generated through a deterministic pipeline with three acquisition branches. The first branch is BIM-native generation, in which the project model is parsed into a standardized object graph with geometry, semantic class, planned schedule links, and work-package membership. The second branch is synthetic sensing generation, in which the BIM model is converted into simulated point clouds under controlled viewpoints, densities, and occlusion settings. The third branch is real-site capture, in which registered scans or digital-twin feeds are attached to the same object graph using a common coordinate frame. The pipeline should export all three branches in one schema so that methods can be trained on synthetic data and tested on real data, or trained and tested within the same modality. [2] [3] [5]
The design follows the logic of reproducible simulation workflows: fixed random seeds, pinned software versions, explicit metadata, and minimal variables that are common across environments. In the construction setting, this means that the benchmark should specify the scan simulator or label generator, the BIM parser version, the registration procedure, and the temporal sampling interval. If multiple sensing stacks are used, the exported variables should be restricted to those that are available in all stacks, so that cross-site comparison does not depend on hidden modality-specific fields. This is especially important when the benchmark spans structured BIM attributes and sparse real-world scans. [5] [2]
A recommended dataset record contains: site identifier, capture time, work package identifier, BIM object set, registered point cloud, semantic component mask, schedule baseline link, uncertainty flags, and provenance tag. The provenance tag should state whether the label is direct, synthetic, inferred, or expert-resolved. Direct labels come from explicit alignment rules; synthetic labels come from BIM-driven generation; inferred labels arise from digital-twin state estimation; expert-resolved labels are used only when a rule cannot resolve ambiguity. This provenance distinction is necessary because benchmark score aggregation should not treat all labels as equally certain. [4] [2]
To support sequential prediction, the pipeline should preserve ordered observation windows rather than only isolated snapshots. The benchmark should declare one fixed base interval per release. All sequences are mapped to that interval by a single default rule: if multiple captures fall within one interval, retain the latest capture and mark earlier captures in that interval as redundant; if no capture falls within an interval, insert an empty observation with a missingness mask and carry forward no geometric evidence. Models may use the masks directly, but the benchmarked target sequence remains aligned to the fixed interval. Variable-length histories are permitted only through left truncation at sequence start; no interpolation of geometry or labels is allowed between observed intervals. This rule keeps irregular site visits comparable without creating synthetic progress events. [1] [3] [5]
5. Standardized evaluation-ready state representation
The benchmark representation should be multi-modal but not redundant. A compact representation is preferable to a large feature dump because comparability requires common inputs. The proposed canonical tensor contains: a geometry channel, a semantic channel, a schedule channel, and an uncertainty channel. Geometry records registered occupancy or component extents. Semantics records class labels and work-package membership. Schedule records planned versus observed state. Uncertainty records capture occlusion, missing components, and label provenance. This structure allows models to operate as classifiers, sequence models, or graph-based estimators while preserving a common interface. [2] [1] [3]
For benchmarking, the main outputs are task-specific. For progress detection, the output is a binary or ordinal label indicating whether progress advanced during the interval. For site state inference, the output is a multi-class state label, such as not-started, partial, advanced, or completed, with class definitions fixed before evaluation. For sequential prediction, the output is the next state or the next transition probability distribution. When a model returns probabilities, the benchmark should score calibration as well as accuracy, because probabilistic outputs are central to digital-twin and Bayesian methods. [1] [4]
The proposed representation supports direct comparison between methods that operate on different modalities. A BIM-only model may use schedule and object-graph channels. A point-cloud model may use geometry and uncertainty channels. A digital-twin model may fuse all channels. This makes the benchmark inclusive without erasing modality-specific strengths. It also prevents unfair comparison by forcing every method to use the same feature extractor or the same segmentation backbone. [2] [3] [1]
6. Comparator classes and scoring protocol
The benchmark should compare at least four method families. The first family is BIM-rule baselines, which infer progress from explicit schedule-object alignment rules. The second is point-cloud segmentation baselines, which infer progress from geometric and semantic recognition. The third is fused multimodal baselines, which combine BIM, point clouds, and digital-twin state variables. The fourth is probabilistic sequence baselines, including Bayesian or LSTM-based temporal models. These comparators reflect the range of methods seen in prior work, where optimization-oriented BIM systems, annotation-free segmentation, and belief-based causal models solve related but distinct problems. [1] [2] [4] [3]
Scoring should be task-specific with one primary metric per task and a fixed aggregation rule. For Task A, progress detection, the primary metric should be macro-averaged F1 over the benchmark labels advanced and not advanced, computed only on non-uncertain cases; a secondary report should include sensitivity to uncertain-case exclusion. For Task B, site state inference, the primary metric should be macro-averaged F1 across the preregistered state classes; object-level component recall should be reported separately as a diagnostic rather than used for ranking. For Task C, sequential progress prediction, the primary metric should be next-state negative log-likelihood when models output probabilities, or macro-F1 on next-state labels when they do not; calibration error should be reported as a mandatory secondary metric for probabilistic models. Scores should first be computed within each work-package family and provenance stratum, then macro-averaged across families, then macro-averaged across provenance strata so that large or label-rich strata do not dominate the ranking. The benchmark should publish all secondary metrics, but the overall selection rule should be based on these declared primary metrics only. [2] [1] [4]
A suitable comparator is not the strongest possible model, but the most interpretable rule set that would be used in practice. For instance, the benchmark should include a baseline that flags progress only when required components cross a fixed completion threshold, because that rule mirrors the operational definition. More complex methods are then judged by whether they improve robustness under occlusion, label scarcity, or schedule drift rather than merely fitting the same threshold more closely. [3] [2]
7. Validation design, failure cases, and boundary conditions
The validation design should be prospective and split by project, not by random frames, so that leakage across adjacent observations does not inflate reported scores. Three transfer settings should be prespecified. The first is cross-site transfer within one building-type family, with the primary estimand defined as the difference between held-out-site and in-family validation macro-F1 for Tasks A and B. The second is cross-type transfer across building-type families that share part of the structural vocabulary, with the primary estimand defined as the held-out target-family macro-F1 together with its calibration error for probabilistic outputs. The third is synthetic-to-real transfer, with the primary estimand defined as the change in task score between synthetic validation data and real held-out data under the same label rule. For sequential prediction under changed observational cadence, the primary estimand should be next-state negative log-likelihood or macro-F1, evaluated after remapping all observations to the benchmark’s fixed base interval. Prior work on BIM-guided synthetic supervision supports testing transfer within similar residential contexts and treating broader transfer to other building types as a separate validation question rather than an assumed property. [2] [5] [1]
Expected failure cases are predictable. Occlusion can hide required components and produce false negatives. Semantic class mismatch can occur when the BIM model omits auxiliary elements that appear in the real scene. Temporal misalignment can make an as-built scan appear consistent with the wrong schedule stage. Sparse point clouds can support segmentation yet remain insufficient for package-level completion. Expert-resolved labels can reduce ambiguity, but they do not remove it; they only make the provenance visible. These are not minor implementation details. They are the primary boundary conditions of the benchmark. [2] [3] [4]
Generalization should therefore be claimed only within stated bounds: a given building type family, a defined work-package taxonomy, and a specified sensing regime. The benchmark should not assume that a model trained on residential structural components will transfer to industrial plants, heavily cluttered fit-outs, or scenes with different semantic taxonomies. Likewise, it should not assume that a sequence predictor trained on one schedule granularity will remain valid at another granularity without redefinition of the temporal unit. [2] [5] [1]
8. Discussion and conclusion
Conceptually, the benchmark design established here resolves three recurrent problems in construction progress tracking evaluation. First, it makes progress state explicit rather than leaving it implicit inside model outputs. Second, it permits reproducible generation and comparison across BIM, point cloud, and digital-twin workflows. Third, it separates label provenance from predictive performance, which is necessary when some labels are synthetic, some are direct, and some are expert-resolved. The proposed design is therefore not a new algorithm but a controlled evaluation environment for algorithms that already exist in related forms. [2] [1] [4] [5]
What remains untested is substantial. The manuscript has not validated the proposed state schema on a live benchmark release, has not compared real numerical performance across methods, and has not established which threshold values are optimal for different task families. It also has not resolved how best to harmonize non-structural components, uncertain schedule updates, or incomplete BIM content in highly heterogeneous projects. Those questions require an empirical implementation phase. The contribution of this manuscript is narrower but more precise: it defines the variables, decision rules, data-generation requirements, and evaluation logic needed to build a reproducible benchmark for construction progress tracking. [2] [3] [1] [4]
Contribution and availability
This manuscript proposes a reproducible benchmark protocol for construction progress tracking that unifies BIM-based, point-cloud-based, and digital-twin-based methods under shared state definitions, deterministic data-generation rules, label provenance tracking, and multi-level evaluation criteria. It translates prior work on BIM-informed prediction, synthetic point-cloud supervision, digital-twin site inference, belief-based uncertainty modeling, and reproducible simulation into a benchmark architecture rather than claiming new empirical performance.
Funding: Roman Vlasov’s own funds. Editorial responsibility and competing interests: Roman Vlasov is the founder and Editor-in-Chief of Smart World Journal and the creator of the BILDRY construction-planning platform. Other competing interests have not been declared.
This conceptual study reports no newly collected empirical data. Its proposed procedures require the prospective validation described above.
References
- Li, Ruixin; Almenweer, Reem A.; Isleem, Haytham F.; Metwally, Ahmed Sayed M. (2026). Optimizing construction risk management through BIM integration and strategic planning. Scientific Reports. 16(1), 28592. doi:10.1038/s41598-026-64364-4.
- Zou, Yiquan; Liang, Tianxiang; Nabi, Jafri Syed Riaz un; Xu, Zhendong; Zhou, Liang; Xiong, Biao (2026). A BIM-Guided Virtual-to-Real Framework for Component-Level Semantic Segmentation of Construction Site Point Clouds. Sensors. 26(1), 308. doi:10.3390/s26010308.
- Qin, Wenli (2026). Application of construction site management based on digital twin and point cloud semantic segmentation technology. PLOS One. 21(3), e0340274. doi:10.1371/journal.pone.0340274.
- Albtoush, Faten; Al-Btoosh, Ja’far A. Aldiabat; Rawashdeh, Taiseer Mustafa; Al-Btoush, Mohammed A. K. A. (2026). Assessing the impact of AI-enabled BIM digital twins on construction risks: a Bayesian Network of Jordanian expert beliefs. Frontiers in Artificial Intelligence. 9, 1846388. doi:10.3389/frai.2026.1846388.
- Domingo, Ferran Aran; Alonso, Pablo Fraile; Torrentó, Josep Rius; Batalla, Oriol Agost; Vilardell, Ignasi Barri; Mayoral, Jordi Vilaplana; Fornés, Jordi Mateo (2026). A reproducible method to generate multi-building, multi-climate HVAC operation datasets with a stochastic exploratory controller. MethodsX. 16, 103866. doi:10.1016/j.mex.2026.103866.