Survey
- Recent Advances in Bayesian Optimization
- Proceeding of IEEE Taking the Human Out of the Loop: A Review of Bayesian Optimization # 3486
- A Survey on High-dimensional Gaussian Process Modeling with Application to Bayesian Optimization
- Expected improvement for expensive optimization: a review
Tutorial
- Exploring Bayesian Optimization
- Bayesian optimization - Martin Krasser's Blog
- How to Implement Bayesian Optimization from Scratch in Python
- Tutorial #8: Bayesian optimization - Borealis AI
- Lecture 12 Bayesian Optimization from CSE 515T: Bayesian Methods in Machine Learning – Fall 2019
- GitHub - krasserm/bayesian-machine-learning: Notebooks about Bayesian methods for machine learning
- [1807.02811] A Tutorial on Bayesian Optimization
- coding book Bayesian Optimization: Theory and Practice Using Python | SpringerLink
CO & Hi-Dim
- NuerIPS 19 # 65 COMBO, Combinatorial Bayesian Optimization using the Graph Cartesian Product
– risky, graph kernal at 2.2, 2.3 worthy reading - ICML # 110 BOCS, Bayesian Optimization of Combinatorial Structures
– math flavor - NuerIPS 19, # 240, TuRBO,Scalable Global Optimization via Local Bayesian Optimization # 230
– improved BOS, engineering flavor - ICML 22, # 25, Casmopolitan, Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces
– [Code] - NuerIPS 22, # 14, Local Latent Space Bayesian Optimization over Structured Inputs
- UAI 21, SAAS, # 39, High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces
- NuerIPS 22, # 6, Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization
- AISTATS 23, #0, BODi, [2303.01774] Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings
– worthy reading
Optional
- GitHub - automl/SMAC3: SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
- OG BO GitHub - fmfn/BayesianOptimization: A Python implementation of global optimization with gaussian processes.
Convergence Rate
- [1101.3501] Convergence rates of efficient global optimization algorithms
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions - ScienceDirect
- https://pubsonline.informs.org/doi/abs/10.1287/opre.2016.1494
- Examples of inconsistency in optimization by expected improvement | SpringerLink
Paper
- NeurIPS 2012 Practical Bayesian Optimization of Machine Learning Algorithms # 7179
- https://hess.copernicus.org/preprints/hess-2017-377/hess-2017-377.pdf
Source Code
- GitHub - pytorch/botorch: Bayesian optimization in PyTorch, # 2.6k stars
- GitHub - fmfn/BayesianOptimization: A Python implementation of global optimization with gaussian processes. # 6.7k stars
- GitHub - pyro-ppl/pyro: Deep universal probabilistic programming with Python and PyTorch, # 8k stars
Gaussian Process
- [the Book!] Gaussian process for machine learning
- A Visual Exploration of Gaussian Processes
-
[2009.10862] An Intuitive Tutorial to Gaussian Processes Regression
– GitHub - jwangjie/Gaussian-Processes-Regression-Tutorial: An Intuitive Tutorial to Gaussian Processes Regression - Implement A Gaussian Process From Scratch | by Shuai Guo | Towards Data Science
- Gaussian Processes, not quite for dummies
- Gaussian-Process-Regression-handout/GPR_handout.pdf at master · NicolasDurrande/Gaussian-Process-Regression-handout · GitHub