Summary of coding reliability and trustworthiness procedures
| Stage | Unit of analysis | % double-coded/n | Metric | Result | Resolution |
|---|---|---|---|---|---|
| First-order (descriptive) | Excerpts within transcript | 24.6% (30/122); 412 excerpts | Cohen’s κ; overall agreement | Initial:κ = 0.66; 82% → Postcalibration: κ = 0.79; 88% | Consensus meetings; third-coder adjudication (11 excerpts; 2.7%) |
| Stability spot-check | Excerpts within transcript | 9.8% (12/122) | Cohen’s κ; overall agreement | κ = 0.81; 90% | No adjudication required |
| Second-order (interpretive) | Themes across cases | – | Negotiated agreement; audit | Codebook v1.0–1.3; decision logs | Team reviews; negative-case analysis |
| Aggregate dimensions | Cross-theme patterns | – | Peer debrief; focus groups | Convergence/divergence/amplification tracked | Revisions to theme boundaries |
| Stage | Unit of analysis | % double-coded/n | Metric | Result | Resolution |
|---|---|---|---|---|---|
| First-order (descriptive) | Excerpts within transcript | 24.6% (30/122); 412 excerpts | Cohen’s κ; overall agreement | Initial:κ = 0.66; 82% | Consensus meetings; third-coder adjudication (11 excerpts; 2.7%) |
| Stability spot-check | Excerpts within transcript | 9.8% (12/122) | Cohen’s κ; overall agreement | κ = 0.81; 90% | No adjudication required |
| Second-order (interpretive) | Themes across cases | – | Negotiated agreement; audit | Codebook v1.0–1.3; decision logs | Team reviews; negative-case analysis |
| Aggregate dimensions | Cross-theme patterns | – | Peer debrief; focus groups | Convergence/divergence/amplification tracked | Revisions to theme boundaries |
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.