Principal Applied Scientist for Copilot Evals

Set the scientific strategy for customer-grounded quality across priority Copilot intents, defining what good means and the tradeoffs across various quality and safety attributes Translate user research, enterprise customer feedback, DSAT, and production incidents into evaluation and post-training priorities, then lead cross-team creation of reusable evaluation, regression, RLE, and post-training assets for the highest-value workflows and failure patterns. Develop methods to assess evaluation-set representativeness, coverage, freshness, discrimination, grader reliability, and alignment with production outcomes, identifying material gaps, drift, and emerging loss patterns. Design behavior and task evaluations, rubrics, graders, and calibration methods that translate qualitative customer expectations into measurable release-over-release quality.