Article navigation
Purpose

Organisations rely on cataloged data quality rules for governance, yet profiler outputs still need screening before promotion to production monitors. This paper studies how semantic plausibility and SQL violation evidence can form an auditable verification score for candidate rules.

Design/methodology/approach

We present Route B, a Bayesian-inspired, prior-adjusted verification score fusing Laplace-smoothed SQL violation rates with hierarchical semantic prior templates. Evaluation uses TPC-H, Financial, and a desensitised Shenzhen open-data catalog (56 candidates). Shared candidate sets are compared with prior-only, SQL-only, confidence-interval, and beta-binomial baselines alongside Route B and optional closed-loop prior correction.

Findings

On the Catalog benchmark, prior-only yields F1 0.33 (36 false promotions) versus Route B F1 0.95 (2 false promotions), cutting proxy review time by about 510 min. Under ambiguous violation rates, hierarchical priors raise F1 from 0.539 to 0.800 versus likelihood-only filtering. On clean warehouses, SQL validation already separates most candidates, and Route B matches SQL-only at precision 1.000 on accepted rules.

Research limitations/implications

Evaluation uses synthetic warehouses and a desensitised catalog; candidates are scored independently; catalog labels follow a written steward protocol; mixed row noise up to 20% does not change warehouse F1.

Practical implications

Route B sits between rule discovery and monitors such as Great Expectations or Deequ as an audit-oriented acceptance layer. A reproduction package regenerates the reported tables without a live LLM endpoint.

Originality/value

An auditable prior-adjusted verification framework with closed-form accept boundaries; a warehouse-plus-catalog verification benchmark from SQL-profiler proposals; and evidence on when semantic–SQL fusion changes decisions relative to SQL-only or prior-only filtering.

Licensed re-use rights only
You do not currently have access to this content.
Don't already have an account? Register

Purchased this content as a guest? Enter your email address to restore access.

Pay-Per-View Access
$41.00
Rental

or Create an Account

Close subscription notice
Close access options