---
@prefix acf: <https://acf-framework.dev/ns/> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

# --- HIGH PRIORITY (ACF v1 scoring) ---

<#M-056> a acf:Measure ;
    acf:id "M-056" ;
    acf:name "domain_count" ;
    rdfs:label "Domain Count" ;
    acf:description "Number of distinct domains where the AI system achieves sufficient knowledge coverage (>80% of domain concepts)" ;
    acf:unit "count" ;
    acf:dataType xsd:integer ;
    acf:collection "automated" ;
    acf:priority "high" ;
    acf:targetVersion "v1" ;
    acf:mapsToDimension <#Breadth> .

<#M-057> a acf:Measure ;
    acf:id "M-057" ;
    acf:name "cross_domain_transfer" ;
    rdfs:label "Cross-Domain Transfer Score" ;
    acf:description "Accuracy on questions requiring knowledge from multiple domains simultaneously" ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "high" ;
    acf:targetVersion "v1" ;
    acf:mapsToDimension <#Breadth>, <#CompositionalGeneralization> .

<#M-058> a acf:Measure ;
    acf:id "M-058" ;
    acf:name "bloom_level_profile" ;
    rdfs:label "Bloom Level Profile" ;
    acf:description "Accuracy stratified by Bloom's taxonomy levels (Remember, Understand, Apply, Analyze, Evaluate, Create). The single most important measure for the Depth dimension." ;
    acf:unit "percent_per_level" ;
    acf:dataType "json" ;
    acf:collection "automated" ;
    acf:priority "critical" ;
    acf:targetVersion "v1" ;
    acf:mapsToDimension <#Depth> .

<#M-059> a acf:Measure ;
    acf:id "M-059" ;
    acf:name "reasoning_chain_validity" ;
    rdfs:label "Reasoning Chain Validity" ;
    acf:description "Percentage of multi-step reasoning chains that are logically valid (each step follows from premises)" ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "llm-graded" ;
    acf:priority "high" ;
    acf:targetVersion "v1" ;
    acf:mapsToDimension <#FormalReasoning> .

<#M-060> a acf:Measure ;
    acf:id "M-060" ;
    acf:name "novel_combination_score" ;
    rdfs:label "Novel Combination Score" ;
    acf:description "Accuracy on tasks requiring novel combinations of known concepts not seen during training" ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "high" ;
    acf:targetVersion "v1" ;
    acf:mapsToDimension <#CompositionalGeneralization> .

<#M-061> a acf:Measure ;
    acf:id "M-061" ;
    acf:name "task_completion_rate" ;
    rdfs:label "Task Completion Rate" ;
    acf:description "Percentage of end-to-end tasks completed successfully to specification without human intervention" ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "high" ;
    acf:targetVersion "v1" ;
    acf:mapsToDimension <#ServiceOrientation> .

# --- ACTION CAPABILITY (ACF v1.1) ---

<#M-067> a acf:Measure ;
    acf:id "M-067" ;
    acf:name "procedure_execution_rate" ;
    rdfs:label "Procedure Execution Rate (PER)" ;
    acf:description "Fraction of known procedures that can be executed on novel input. PER = procedures attempted / procedures in knowledge." ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "high" ;
    acf:targetVersion "v1.1" ;
    acf:mapsToDimension <#ActionCapability> .

<#M-068> a acf:Measure ;
    acf:id "M-068" ;
    acf:name "step_completion_rate" ;
    rdfs:label "Step Completion Rate (SCR)" ;
    acf:description "Fraction of procedure steps completed correctly. SCR = steps correct / steps attempted." ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "high" ;
    acf:targetVersion "v1.1" ;
    acf:mapsToDimension <#ActionCapability> .

<#M-069> a acf:Measure ;
    acf:id "M-069" ;
    acf:name "answer_correctness_rate" ;
    rdfs:label "Answer Correctness Rate (ACR)" ;
    acf:description "Fraction of executed procedures producing correct final output. ACR = correct outputs / procedures completed." ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "high" ;
    acf:targetVersion "v1.1" ;
    acf:mapsToDimension <#ActionCapability> .

# --- MEDIUM PRIORITY (ACF v2) ---

<#M-062> a acf:Measure ;
    acf:id "M-062" ;
    acf:name "response_appropriateness" ;
    rdfs:label "Response Appropriateness" ;
    acf:description "Degree to which response complexity and vocabulary matches the user's expertise level" ;
    acf:unit "score_1_5" ;
    acf:dataType xsd:decimal ;
    acf:collection "llm-graded" ;
    acf:priority "medium" ;
    acf:targetVersion "v2" ;
    acf:mapsToDimension <#ServiceOrientation> .

<#M-063> a acf:Measure ;
    acf:id "M-063" ;
    acf:name "domain_adjacent_score" ;
    rdfs:label "Domain-Adjacent Query Handling" ;
    acf:description "Accuracy in correctly handling queries at the boundary of known domains without fabricating information" ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "medium" ;
    acf:targetVersion "v2" ;
    acf:mapsToDimension <#GeneralizationBoundaryAwareness> .

<#M-064> a acf:Measure ;
    acf:id "M-064" ;
    acf:name "graceful_degradation" ;
    rdfs:label "Graceful Degradation Score" ;
    acf:description "Quality of the system's response when operating at the boundary of its knowledge (informative failure vs. hallucination)" ;
    acf:unit "score_0_1" ;
    acf:dataType xsd:decimal ;
    acf:collection "llm-graded" ;
    acf:priority "medium" ;
    acf:targetVersion "v2" ;
    acf:mapsToDimension <#GeneralizationBoundaryAwareness> .

<#M-065> a acf:Measure ;
    acf:id "M-065" ;
    acf:name "self_generated_goals_pct" ;
    rdfs:label "Self-Generated Learning Goals" ;
    acf:description "Percentage of learning objectives that were self-generated by the system rather than externally assigned" ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "medium" ;
    acf:targetVersion "v2" ;
    acf:mapsToDimension <#Autonomy> .

<#M-066> a acf:Measure ;
    acf:id "M-066" ;
    acf:name "self_correction_rate" ;
    rdfs:label "Self-Correction Rate" ;
    acf:description "Percentage of errors detected and corrected by the system without human intervention" ;
    acf:unit "percent" ;
    acf:dataType xsd:decimal ;
    acf:collection "automated" ;
    acf:priority "medium" ;
    acf:targetVersion "v2" ;
    acf:mapsToDimension <#Autonomy> .
---

# Proposed Measures: M-056 through M-066

**Status:** Proposed -- pending validation and implementation
**Source:** ACF gap analysis of original 55 measures (M-001 through M-055) against the 10 ACF dimensions

## Origin and Rationale

These 11 measures were identified through a systematic gap analysis of the original ACF measure set (M-001 through M-055) against the 9 ACF dimensions (Breadth, Depth, Formal Reasoning, Factual Grounding, Knowledge Transparency, Compositional Generalization, Service Orientation, Generalization Boundary Awareness, and Autonomy). The analysis revealed that several dimensions lacked sufficient measure coverage to produce reliable dimension scores, making it impossible to compute a meaningful ACF certification level for those dimensions.

The gap analysis found that while some dimensions (e.g., Factual Grounding, Knowledge Transparency) had strong coverage from existing measures, others -- particularly Breadth, Depth, Compositional Generalization, and Autonomy -- had either no dedicated measures or relied on indirect proxies that captured only a fraction of the dimension's intent.

## Priority Levels

Measures are assigned one of three priority levels:

### Critical (1 measure)

**M-058 (Bloom Level Profile)** is the single highest-priority addition in this set. The Depth dimension is defined entirely around Bloom's taxonomy levels (L1 Remember through L6 Create), yet no existing measure directly captures performance stratified by Bloom level. Without M-058, the Depth dimension score must be inferred from aggregate accuracy -- an approach that collapses six distinct cognitive levels into a single number and loses the core signal the dimension is designed to capture. Implementing M-058 unlocks the entire L1-L6 sub-level scoring rubric defined in `knowledge/dimensions/depth.md`.

### High (5 measures: M-056, M-057, M-059, M-060, M-061)

High-priority measures are those required for ACF v1 scoring. Each fills a gap where a dimension either lacks any direct measure or where the existing measures provide insufficient discrimination. These six measures (including M-058) collectively enable scoring across all 9 dimensions, making it possible to compute a complete ACF certification level.

### Medium (5 measures: M-062, M-063, M-064, M-065, M-066)

Medium-priority measures refine dimensions that already have partial coverage from the high-priority set or from existing measures. They are targeted for ACF v2 and improve scoring granularity without blocking the initial certification workflow.

## Measure Details and Collection Guidance

### M-056: Domain Count

| Property | Value |
|----------|-------|
| Dimension | Breadth |
| Unit | count (integer) |
| Collection | Automated |
| Priority | High |

**What it measures:** The number of distinct knowledge domains where the AI system achieves sufficient coverage, defined as covering more than 80% of domain concepts.

**How to collect:** For each domain in the system's curriculum, compute topic coverage as the ratio of concepts the system can correctly address to total concepts defined for that domain. Count the number of domains where this ratio exceeds 0.80. This requires a well-defined curriculum with per-domain concept inventories.

**Why it matters:** Breadth cannot be scored without knowing how many domains the system has meaningfully learned. Raw topic counts or total knowledge volume do not capture breadth -- what matters is the number of domains where coverage is sufficient to be useful.

---

### M-057: Cross-Domain Transfer Score

| Property | Value |
|----------|-------|
| Dimensions | Breadth, Compositional Generalization |
| Unit | percent (decimal) |
| Collection | Automated |
| Priority | High |

**What it measures:** Accuracy on questions that require the system to draw on knowledge from two or more domains simultaneously.

**How to collect:** Construct a test battery of questions explicitly designed to require cross-domain knowledge. Each question should be tagged with the domains it draws upon. Score the system on this battery and compute overall accuracy. Questions should be designed so that single-domain knowledge is insufficient for a correct answer.

**Why it matters:** Cross-domain transfer is the hallmark of Breadth sub-levels B3 and B4. It also contributes to Compositional Generalization by testing whether the system can combine knowledge elements from different sources. Without this measure, there is no way to distinguish a system that knows many domains in isolation from one that can integrate them.

---

### M-058: Bloom Level Profile

| Property | Value |
|----------|-------|
| Dimension | Depth |
| Unit | percent per level (JSON) |
| Collection | Automated |
| Priority | Critical |

**What it measures:** Accuracy stratified across all six Bloom's taxonomy levels: Remember (L1), Understand (L2), Apply (L3), Analyze (L4), Evaluate (L5), and Create (L6).

**How to collect:** Tag each question in the exam battery with its target Bloom level. After evaluation, compute accuracy separately for each level. The result is a JSON object with six keys (L1 through L6), each mapping to a percentage score.

Example output format:
```json
{
  "L1_remember": 95.0,
  "L2_understand": 88.5,
  "L3_apply": 72.0,
  "L4_analyze": 61.3,
  "L5_evaluate": 45.8,
  "L6_create": 28.2
}
```

**Why it matters:** This is the single most important proposed measure. The Depth dimension is structured entirely around Bloom's taxonomy, with sub-levels L1 through L6 mapping directly to Bloom's levels. Without per-level accuracy data, the Depth dimension cannot be scored at its intended granularity. A system that scores 70% overall might achieve 95% on Remember but 20% on Create -- a profile that demands a very different Depth score than a system scoring 70% uniformly across all levels. M-058 makes this distinction visible.

---

### M-059: Reasoning Chain Validity

| Property | Value |
|----------|-------|
| Dimension | Formal Reasoning |
| Unit | percent (decimal) |
| Collection | LLM-graded |
| Priority | High |

**What it measures:** The percentage of multi-step reasoning chains produced by the system that are logically valid -- meaning each step follows from its premises and the chain as a whole is sound.

**How to collect:** Present the system with problems requiring multi-step reasoning. Capture the system's reasoning chain (chain-of-thought, intermediate steps, or explicit reasoning trace). Use an LLM grader (or human evaluator) to assess each chain for logical validity: Does each step follow from the previous? Are there logical fallacies? Is the overall argument sound? Report the percentage of chains rated as valid.

**Why it matters:** Formal Reasoning requires more than arriving at correct answers -- it requires arriving at them for the right reasons through valid logical steps. Existing accuracy measures may reward systems that reach correct conclusions through flawed reasoning (lucky guesses, pattern matching). M-059 specifically targets the quality of the reasoning process itself.

---

### M-060: Novel Combination Score

| Property | Value |
|----------|-------|
| Dimension | Compositional Generalization |
| Unit | percent (decimal) |
| Collection | Automated |
| Priority | High |

**What it measures:** Accuracy on tasks that require novel combinations of known concepts -- combinations the system has not encountered during training.

**How to collect:** Identify concept pairs or groups that appear in the system's training data individually but never together. Construct questions that require combining these concepts in novel ways. Score the system's accuracy on these questions. The key design criterion is that both individual concepts must be well-learned (verified by high single-concept accuracy) while the combination is genuinely novel.

**Why it matters:** Compositional generalization is the ability to compose known building blocks in new ways. This is distinct from memorization (reproducing known combinations) and from cross-domain transfer (combining knowledge across domain boundaries). M-060 specifically tests whether the system can generalize combinatorially -- a core indicator of flexible, compositional intelligence.

---

### M-061: Task Completion Rate

| Property | Value |
|----------|-------|
| Dimension | Service Orientation |
| Unit | percent (decimal) |
| Collection | Automated |
| Priority | High |

**What it measures:** The percentage of end-to-end tasks that the system completes successfully to specification without requiring human intervention.

**How to collect:** Define a set of representative tasks with clear success criteria. Present each task to the system and allow it to work through to completion (or failure/timeout). Score each task as complete (meets all success criteria) or incomplete. Report the percentage of tasks completed. Tasks should span the range of complexity the system is expected to handle and should be evaluated against predefined rubrics rather than subjective judgment.

**Why it matters:** Service Orientation captures whether the system can actually deliver useful outcomes to its users. Existing measures may assess knowledge accuracy or reasoning quality in isolation, but a system that knows the right answer yet fails to deliver it in a usable form scores poorly on Service Orientation. M-061 provides a direct, outcome-based signal.

---

### M-062: Response Appropriateness

| Property | Value |
|----------|-------|
| Dimension | Service Orientation |
| Unit | score 1--5 (decimal) |
| Collection | LLM-graded |
| Priority | Medium |

**What it measures:** The degree to which the system's response complexity and vocabulary matches the user's expertise level.

**How to collect:** Present the system with queries from users at different expertise levels (novice, intermediate, expert). Use an LLM grader to rate each response on a 1--5 scale for appropriateness: Does it use vocabulary the user would understand? Is the level of detail suitable? Does it avoid being either condescending (over-simplifying for experts) or overwhelming (jargon-heavy for novices)?

**Why it matters:** A system that gives technically correct but incomprehensible responses is not service-oriented. This measure captures the "meeting the user where they are" aspect of Service Orientation, complementing M-061's focus on task completion.

---

### M-063: Domain-Adjacent Query Handling

| Property | Value |
|----------|-------|
| Dimension | Generalization Boundary Awareness (GBA) |
| Unit | percent (decimal) |
| Collection | Automated |
| Priority | Medium |

**What it measures:** Accuracy in correctly handling queries at the boundary of the system's known domains, without fabricating information.

**How to collect:** Construct queries that sit at the edge of the system's trained domains -- topics that are related to but not covered by its training data. Evaluate whether the system correctly identifies these as outside its confident knowledge and handles them appropriately (e.g., acknowledging uncertainty, redirecting, providing caveated partial answers) rather than hallucinating authoritative-sounding responses.

**Why it matters:** GBA is fundamentally about knowing the limits of one's knowledge. M-063 tests boundary behavior directly by probing the fuzzy edges where knowledge trails off. Systems that fabricate confident answers for boundary queries score poorly; systems that gracefully acknowledge their limits score well.

---

### M-064: Graceful Degradation Score

| Property | Value |
|----------|-------|
| Dimension | Generalization Boundary Awareness (GBA) |
| Unit | score 0--1 (decimal) |
| Collection | LLM-graded |
| Priority | Medium |

**What it measures:** The quality of the system's response when operating at or beyond the boundary of its knowledge -- specifically, whether it fails informatively (explaining what it does and does not know) rather than failing silently or through hallucination.

**How to collect:** Present the system with queries it cannot fully answer (verified through prior testing). Use an LLM grader to score the response on a 0--1 scale: 0 for confident hallucination, 0.5 for unhelpful refusal, and 1.0 for an informative, well-calibrated response that clearly communicates what the system knows, what it does not, and where the user might find better information.

**Why it matters:** This complements M-063 by assessing not just whether the system detects boundary cases but how well it communicates its limitations. A system that says "I don't know" is better than one that hallucinates, but a system that says "I don't know X, but I do know Y which may help, and you might find X in Z" is better still.

---

### M-065: Self-Generated Learning Goals

| Property | Value |
|----------|-------|
| Dimension | Autonomy |
| Unit | percent (decimal) |
| Collection | Automated |
| Priority | Medium |

**What it measures:** The percentage of the system's learning objectives that were self-generated rather than externally assigned by a human operator or predefined curriculum.

**How to collect:** Track the source of each learning objective during the system's training or operation cycle. Classify each objective as externally assigned (from curriculum, human instruction, or configuration) or self-generated (identified by the system through gap analysis, curiosity-driven exploration, or metacognitive reflection). Report the ratio of self-generated objectives to total objectives.

**Why it matters:** Autonomy in the ACF context includes the ability to direct one's own learning. A system that only learns what it is told to learn has limited autonomy regardless of how well it learns. M-065 provides a direct measure of self-directed learning behavior, which is a prerequisite for higher Autonomy sub-levels.

---

### M-066: Self-Correction Rate

| Property | Value |
|----------|-------|
| Dimension | Autonomy |
| Unit | percent (decimal) |
| Collection | Automated |
| Priority | Medium |

**What it measures:** The percentage of errors that the system detects and corrects on its own, without human intervention or external feedback.

**How to collect:** Track errors during system operation (incorrect answers, failed tasks, logical inconsistencies). For each error, record whether it was detected and corrected by the system itself (through self-monitoring, re-evaluation, or metacognitive processes) or whether it required external feedback to identify. Report the ratio of self-corrected errors to total errors.

**Why it matters:** Self-correction is a hallmark of autonomous, self-improving systems. A system that can identify and fix its own mistakes demonstrates both metacognitive awareness (knowing it made an error) and autonomous agency (acting to correct it). This measure directly captures a key component of the Autonomy dimension that is not covered by any existing measure.

## Dimension Coverage Summary

The following table shows which ACF dimensions these proposed measures address:

| Dimension | Proposed Measures | Priority |
|-----------|-------------------|----------|
| Breadth | M-056, M-057 | High |
| Depth | M-058 | Critical |
| Formal Reasoning | M-059 | High |
| Compositional Generalization | M-057, M-060 | High |
| Service Orientation | M-061, M-062 | High, Medium |
| Generalization Boundary Awareness | M-063, M-064 | Medium |
| Autonomy | M-065, M-066 | Medium |
| Factual Grounding | -- (adequate existing coverage) | -- |
| Knowledge Transparency | -- (adequate existing coverage) | -- |
| **Action Capability** | **M-067, M-068, M-069** | **High (v1.1)** |

## Implementation Notes

- **Automated** measures (M-056, M-057, M-058, M-060, M-061, M-063, M-065, M-066) can be computed programmatically from exam results, system logs, or curriculum metadata. These should be prioritized for implementation as they require no manual evaluation.
- **LLM-graded** measures (M-059, M-062, M-064) require a separate evaluator (LLM or human) to assess response quality. These introduce a dependency on evaluator availability and inter-rater reliability and should include calibration procedures.
- M-058 produces a JSON object rather than a scalar value. Scoring infrastructure must handle structured measure values and map them to the per-sub-level scores defined in `knowledge/dimensions/depth.md`.
