---
title: "Summary: Three Models of RLHF Annotation: Extension, Evidence, and Authority"
date: 2026-04-28
tags: ['paper', 'research', 'ai']
---
# Summary: Three Models of RLHF Annotation: Extension, Evidence, and Authority


**Source**: [Original Paper](http://arxiv.org/abs/2604.25895v1)
Saved: 2026-05-08 03:29
Source: 2026-04-28_17-39-14Z_ThreeModelsofRLHFAnnotation_Extension_Evidence_and.md

---

## Summary
Argues that RLHF annotations play three distinct normative roles: extension, evidence, and authority. The paper analyzes how those roles affect solicitation, validation, and aggregation, and recommends separating annotation dimensions instead of forcing a single unified pipeline.

## Semantic links
- [[concepts/papers/2026-06-10_14-38-23Z_TowardsResponsiblyNon_CompliantMachines_summary.md|Summary: 2026-06-10_14-38-23Z_TowardsResponsiblyNon_CompliantMachines.md]] — 1 title term overlap; shared tags: ai, paper, research; 8 summary/topic terms overlap

## Key Takeaways
- Makes the normative role of annotator judgments explicit.
- Surveys RLHF and related methods through the three-model lens.
- Highlights failure modes from conflating extension, evidence, and authority.

## Context
The paper is a conceptual analysis of preference-based alignment pipelines.

## Implications
Separating annotation roles could make RLHF design more principled and easier to validate.

## Original Reference
- Title: Three Models of RLHF Annotation: Extension, Evidence, and Authority
- Authors: Steve Coyne
- Published: 2026-04-28T17:39:14Z
- URL: http://arxiv.org/abs/2604.25895v1
- Source file: /home/rich/wiki/ai-research/raw/papers/2026-04-28_17-39-14Z_ThreeModelsofRLHFAnnotation_Extension_Evidence_and.md

## Related Concepts

- [[concepts/search-retrieval/search-retrieval-hub.md|Search Retrieval Hub]]
- [[concepts/data-curation/data-curation-hub.md|Data Curation Hub]]
- [[concepts/alignment-safety/alignment-hub.md|Alignment Hub]]
- [[concepts/training-optimization/training-optimization-hub.md|Training Optimization Hub]]