Activity v1
Evaluation Strategy Designer
Assemble a balanced evaluation plan from risks cases metrics review and release gates.
rubric
beginner
seed 11Evaluation Strategy Designer: Baseline
Explore the core controls with stable inputs and visible assumptions.
Assumptions
The simulation is deterministic and intentionally simplifies provider and hardware behavior.
Failure injection
Stable baseline with no injected production fault.
Evaluation plan
Coverage and confidence are rubric prompts, not statistical guarantees. Real confidence depends on sampling, labels, judge validity, and task distribution.
Dimension coverage33%
Planning confidence73%
Human-reviewed cases10
Recommended gate
Require no critical security failures, explicit minimums for each selected dimension, slice-level checks, and a cost/latency budget. Keep a protected holdout and turn production incidents into new cases.
0 saved attempts
Expected outcomes
- Choose representative evaluations
- Define measurable release criteria