List hyperparameter tuning methods

Explain widely used approaches for hyperparameter tuning in machine learning. For each approach, cover:

• The conceptual idea behind how it works.
• Its strengths and weaknesses, such as efficiency, how easily it parallelizes, and sample efficiency. At minimum, include manual search, grid search, random search, and more advanced methods like Bayesian optimization or adaptive schemes. Overview: This question tests knowledge of hyperparameter tuning methods, including the ability to compare different search strategies, weigh trade-offs in efficiency and parallelization, and design experiments to improve model performance.