Budget-Aware Hyperparameter Optimization: Selecting HPO Strategies Under Parallelism and a Fixed Compute Budget

Hyperparameter optimization methods are typically benchmarked one trial at a time, but real compute clusters run many trials in parallel — which changes how these methods perform in ways the standard comparisons miss. This project investigates whether the choice of HPO method should depend on the computational setup (number of parallel workers, remaining budget), and whether a system could select an appropriate method automatically rather than committing to one in advance. Early exploratory runs suggest this is worth pursuing, but the data needs further validation before drawing conclusions.

The work involves running controlled experiments, analyzing results statistically, and building a lightweight selection system, with the goal of producing a manuscript and a practical guide for practitioners.