ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning
Published in International Conference on Machine Learning (ICML), 2026
* Equal contribution
ACTIVEULTRAFEEDBACK introduces a modular active learning pipeline that uses uncertainty-aware reward estimates to select informative response pairs, reducing the amount of preference data needed for strong downstream performance.
