Methodology
Qualitative comparative documentary analysis with AI-assisted coding validation
Research Design
The survey employs a qualitative comparative documentary analysis structured in three phases. Twelve regulatory documents across 10 jurisdictions are systematically coded against the PPP framework, producing 170 classifications that are independently validated through an AI-assisted coding pipeline.
Phase 1
Document Collection
Assemble the primary corpus of 12 regulatory instruments across 10 jurisdictions, selected for regulatory significance, typological diversity, and geographic coverage.
Phase 2
Structured Coding
Apply the PPP coding scheme (17 sub-dimensions) to each document. Rate each as Mandatory, Recommended, or Absent with provision-level evidence.
Phase 3
Comparative Analysis
Construct a jurisdiction × sub-dimension matrix. Analyze within-jurisdiction profiles, cross-jurisdictional patterns, and interdependencies.
Document Selection Criteria
| Criterion | Inclusion | Exclusion |
|---|---|---|
| Document type | Binding legislation, executive directives, official policy frameworks, intergovernmental standards | Draft proposals, academic commentary, media reports |
| Temporal scope | Instruments adopted or in force as of early 2026 | Expired or fully superseded instruments |
| Language | English originals or official translations | Documents without authoritative English versions |
Coding Procedure
Each regulatory document was coded against all 17 PPP sub-dimensions. For each sub-dimension, the coding captured:
Presence / Absence
Whether the regulatory instrument addresses this sub-dimension.
Regulatory Intensity
Mandatory (binding obligation), Recommended (encouraged but not binding), or Absent (not addressed).
Provision Detail
Specific articles, sections, or clauses corresponding to the sub-dimension.
Textual Summary
Brief description of the provision’s content and scope.
AI-Assisted Coding: Validation Experiment
All 170 classifications (17 sub-dimensions × 10 jurisdictions) were independently coded by a large language model and compared against manual expert coding. This serves as both a validation instrument and a methodological contribution demonstrating the viability of AI-augmented regulatory analysis.
87.6%
Overall agreement across 170 classifications
Agreement by Jurisdiction
| Jurisdiction | Agree | Disagree | Rate |
|---|---|---|---|
| EU AI Act | 17/17 | 0 | 100.0% |
| Japan AI Promotion Act | 16/17 | 1 | 94.1% |
| OECD AI Principles | 16/17 | 1 | 94.1% |
| UNESCO Recommendation | 16/17 | 1 | 94.1% |
| Colorado AI Act | 15/17 | 2 | 88.2% |
| China (combined) | 15/17 | 2 | 88.2% |
| Singapore Framework | 15/17 | 2 | 88.2% |
| NIST AI RMF 1.0 | 13/17 | 4 | 76.5% |
| Canada DADM | 13/17 | 4 | 76.5% |
| UK White Paper | 13/17 | 4 | 76.5% |
| Overall | 149/170 | 21 | 87.6% |
Agreement by Legal Form
Agreement by PPP Dimension
Disagreement Patterns (21 disagreements)
AI overrates coverage
The LLM classified provisions as present where the manual coder judged them too tangential or indirect to constitute substantive coverage.
AI underrates coverage
The LLM coded provisions as Absent where the manual coder found them indirectly present, particularly for infrastructure standards (PL2).
Intensity confusion
The LLM disagreed on the Mandatory/Recommended boundary for provisions using strong normative language without explicit penalties.
Key finding: Agreement is predicted by regulatory text legibility (the degree to which obligations specify identifiable duty-holders, enumerated actions, and explicit conditions) rather than by document complexity, subject matter, or jurisdiction. The most complex document (EU AI Act, 113 articles) achieves 100% agreement; shorter voluntary frameworks achieve 76.5%.
Methodological Limitations
- •Single-coder design constrains inter-coder reliability; mitigated by documented coding decisions with provision-level evidence and AI-assisted validation (87.6% agreement).
- •The three-level coding scale (Mandatory / Recommended / Absent) collapses substantial within-category variation. The EU’s five-article GPAI regime and China’s single-regulation approach both receive a Mandatory rating.
- •The survey captures regulatory design as of early 2026, not implementation or enforcement in practice.
- •Reliance on English-language documents may not fully capture the intent of instruments originally drafted in other languages, particularly Chinese regulatory texts.
- •The ten-jurisdiction scope excludes significant regulatory activity in Latin America, South Asia, and Africa.