Articles

Here is a commented bibliography on unbiased MCMC.

For a recent review on the topic, see Atchadé and Jacob (2024), Unbiased Markov Chain Monte Carlo: what, why and how, available on arXiv, a version of which will hopefully be a chapter in the upcoming 2nd edition of the Handbook of Markov chain Monte Carlo.

Core ideas of unbiased estimators using couplings

• Glynn and Rhee (2014) : first paper presenting the idea that contractive couplings of Markov chains, combined with a telescopic sum argument, lead to implementable unbiased estimators of expectations with respect to the stationary distribution of these Markov chains.

• Agapiou, Roberts, and Vollmer (2018) : paper applying these ideas to the setting of Markov chain Monte Carlo methods, with contractive couplings.

• Vihola (2018) : presents a general unbiased scheme with an explicit expression for its variance.

• Pierre E. Jacob, O’Leary, and Atchadé (2020) and its discussion : paper applying these ideas with couplings that induce exact meetings, and with modifications that make the unbiased estimators nearly as efficient as the original MCMC estimators when tuned properly.

Related ideas and improvements

• McLeish (2011) : an attempt at bias removal from MCMC estimators via a telescopic sum trick

• Craiu and Meng (2022) : paper introduce control variates improvements of the estimators in Pierre E. Jacob, O’Leary, and Atchadé (2020).

• Douc et al. (2025+) : connection between Glynn & Rhee’s unbiased estimators and the solutions of the Poisson equation, and novel estimators of the asymptotic variance in the Markov chain Central Limit Theorem. Also provides clean conditions under which unbiased MCMC estimators have finite \(p\) moments.

• Neal (1999), also Vanetti and Doucet (2020) : alternative construction for nearly-unbiased estimators using couplings.

Convergence diagnostics

• Biswas, Jacob, and Vanetti (2019) : develops an idea presented at the end of Pierre E. Jacob, O’Leary, and Atchadé (2020) to obtain bounds on the total variation distance and the 1-Wasserstein distance between a Markov chain at fixed time \(t\) and its limiting distribution.

• Johnson (1996), Johnson (1998) : propose convergence diagnostics based on couplings, including couplings that induce exact meetings for random walk Metropolis-Rosenblut-Teller-Hastings algorithms.

• Corenflos and Dau (2025) : proposes computable bounds on the f-divergence (e.g. KL, TV or \(\chi^2\)) between the target distribution \(\pi\) and the marginal distribution of the chain at time \(t\). The method involves running \(N\) pairs of chains up to time \(t\), coupled such that pairs can meet exactly at each iteration. Each chain is weighted and the weights are harmonized as the pairs coalesce. The matching between chains is randomized at each time step, so that all chains eventually interact with one another.

Normalizing constant estimation

• Rischard, Jacob, and Pillai (2018) : starts from thermodynamic integration, with a path of distributions \((\pi_\lambda)\) linking a tractable \(\pi_0\) and the target \(\pi\). Then randomizes \(\lambda\) and runs unbiased MCMC for \(\pi_\lambda\), to ultimately unbiased estimators of log ratios of normalizing constants.

Estimating nonlinear functions of expectations

• Wang and Wang (2022)

Specific classes of MCMC algorithms

Metropolis-Rosenbluth-Teller-Hastings

• Wang, O’Leary, and Jacob (2021), O’Leary and Wang (2024) : construction of one-step maximal coupling of general MRTH kernels, and characterization of the set of all couplings of these kernels.

• Papp and Sherlock (2024) : propose couplings of random walk proposal MRTH that are optimal for a class of targets in high-dimension, improving upon the reflection couplings employed in Pierre E. Jacob, O’Leary, and Atchadé (2020).

• Deligiannidis et al. (2025) : the case of independent proposals, with a possibly unbounded ratio target/proposal. The common draws coupling of independent proposal MRTH is maximal. Connects with the idea of debiasing self-normalised importance sampling, as initiated in Middleton et al. (2019). Also looks at interconnection between unbiased MCMC and robust mean estimation.

Pseudo-marginal methods and conditional particle filters

• Middleton et al. (2019), Middleton et al. (2020)

• Pierre E. Jacob, Lindsten, and Schön (2020) : paper applying Glynn & Rhee’s idea to conditional particle filters, for which simple couplings lead to exact meetings. First version was in Pierre E. Jacob, Lindsten, and Schön (2016).

• Lee, Singh, and Vihola (2020)

• Karjalainen et al. (2023)

Gradient-based MCMC

• Heng and Jacob (2019), Xu et al. (2021)

• Chada et al. (2024)

Gibbs sampling

• Bacallado et al. (2021)

• Nguyen, Trippe, and Broderick (2022)

• Biswas et al. (2022)

• Ceriani and Zanella (2024)

• Ascolani and Zanella (2025) : Gibbs sampler high-dimensional probit regression. The paper obtains results about the mixing time of the sampler, not using couplings; but coupling-based total variation upper bounds are obtained to verify the theory in numerical experiments.

• Du and He (2024)

Piecewise deterministic Markov chain Monte Carlo

• Corenflos, Sutton, and Chopin (2023)

Applications and Other

• Kelly, Ryder, and Clarté (2023) : phylogenetic inference

• Ruiz et al. (2021) : unbiased MCMC for gradient estimation in variational auto-encoders

• Zhu and Atchadé (2020) :

• Hou, Wang, and Atchadé (2024) : application to a Gibbs sampler for variable selection in high-dimension

References

Agapiou, Sergios, Gareth O Roberts, and Sebastian J Vollmer. 2018. “Unbiased Monte Carlo: Posterior Estimation for Intractable/Infinite-Dimensional Models.” Bernoulli 24 (3): 1726–86.

Ascolani, Filippo, and Giacomo Zanella. 2025. “Mixing Times of Data-Augmentation Gibbs Samplers for High-Dimensional Probit Regression.” arXiv Preprint arXiv:2505.14343.

Atchadé, Yves F, and Pierre E Jacob. 2024. “Unbiased Markov Chain Monte Carlo: what, why, and how.” arXiv Preprint arXiv:2406.06851.

Bacallado, Sergio, Stefano Favaro, Samuel Power, and Lorenzo Trippa. 2021. “Perfect Sampling of the Posterior in the Hierarchical Pitman–Yor Process.” Bayesian Analysis 17 (3): 685.

Biswas, Niloy, Anirban Bhattacharya, Pierre E. Jacob, and James E. Johndrow. 2022. “Coupling-Based Convergence Assessment of Some Gibbs Samplers for High-Dimensional Bayesian Regression with Shrinkage Priors.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 84 (3): 973–96.

Biswas, Niloy, Pierre E Jacob, and Paul Vanetti. 2019. “Estimating Convergence of Markov Chains with L-Lag Couplings.” In Advances in Neural Information Processing Systems, 7389–99.

Ceriani, Paolo Maria, and Giacomo Zanella. 2024. “Linear-Cost Unbiased Posterior Estimates for Crossed Effects and Matrix Factorization Models via Couplings.” arXiv Preprint arXiv:2410.08939.

Chada, Neil K., Benedict Leimkuhler, Daniel Paulin, and Peter A. Whalley. 2024. “Unbiased Kinetic Langevin Monte Carlo with Inexact Gradients.” https://arxiv.org/abs/2311.05025.

Corenflos, Adrien, and Hai-Dang Dau. 2025. “A coupling-based approach to f-divergences diagnostics for Markov chain Monte Carlo.” arXiv Preprint arXiv:2510.07559.

Corenflos, Adrien, Matthew Sutton, and Nicolas Chopin. 2023. “Debiasing Piecewise Deterministic Markov Process Samplers Using Couplings.” arXiv Preprint arXiv:2306.15422.

Craiu, Radu V, and Xiao-Li Meng. 2022. “Double Happiness: Enhancing the Coupled Gains of L-Lag Coupling via Control Variates.” Statistica Sinica 32: 1–22.

Deligiannidis, George, Pierre E. Jacob, El Mahdi Khribch, and Guanyang Wang. 2025. “On Importance Sampling and Independent Metropolis-Hastings with an Unbounded Weight Function.” arXiv Preprint arXiv:2411.09514. https://arxiv.org/abs/2411.09514.

Douc, Randal, Pierre E Jacob, Anthony Lee, and Dootika Vats. 2025+. “Solving the Poisson Equation Using Coupled Markov Chains.” arXiv Preprint arXiv:2206.05691, 2025+.

Du, Jiarui, and Zhijian He. 2024. “Unbiased Markov Chain Quasi-Monte Carlo for Gibbs Samplers.” arXiv Preprint arXiv:2403.04407.

Glynn, Peter W, and Chang-Han Rhee. 2014. “Exact Estimation for Markov Chain Equilibrium Expectations.” Journal of Applied Probability 51 (A): 377–89.

Heng, Jeremy, and Pierre E Jacob. 2019. “Unbiased Hamiltonian Monte Carlo with Couplings.” Biometrika 106 (2): 287–302.

Hou, Tianrui, Liwei Wang, and Yves Atchadé. 2024. “Laplace Approximation for Bayesian Variable Selection via Le Cam’s One-Step Procedure.” arXiv Preprint arXiv:2407.20580. https://arxiv.org/abs/2407.20580.

Jacob, Pierre E., Fredrik Lindsten, and Thomas B. Schön. 2016. “Coupling of Particle Filters.” arXiv Preprint arXiv:1606.01156. https://arxiv.org/abs/1606.01156.

Jacob, Pierre E, Fredrik Lindsten, and Thomas B Schön. 2020. “Smoothing with Couplings of Conditional Particle Filters.” Journal of the American Statistical Association 115 (530): 721–29.

Jacob, Pierre E, John O’Leary, and Yves F Atchadé. 2020. “Unbiased Markov Chain Monte Carlo Methods with Couplings.” Journal of the Royal Statistical Society Series B (with Discussion) 82 (3): 543–600.

Johnson, Valen E. 1996. “Studying Convergence of Markov Chain Monte Carlo Algorithms Using Coupled Sample Paths.” Journal of the American Statistical Association 91 (433): 154–66.

———. 1998. “A Coupling-Regeneration Scheme for Diagnosing Convergence in Markov Chain Monte Carlo Algorithms.” Journal of the American Statistical Association 93 (441): 238–48.

Karjalainen, Joona, Anthony Lee, Sumeetpal S. Singh, and Matti Vihola. 2023. “Mixing Time of the Conditional Backward Sampling Particle Filter.” arXiv Preprint arXiv:2312.17572.

Kelly, Luke J, Robin J Ryder, and Grégoire Clarté. 2023. “Lagged Couplings Diagnose Markov Chain Monte Carlo Phylogenetic Inference.” Annals of Applied Statistics (to Appear).

Lee, Anthony, Sumeetpal S. Singh, and Matti Vihola. 2020. “Coupled conditional backward sampling particle filter.” The Annals of Statistics 48 (5): 3066–89. https://doi.org/10.1214/19-AOS1922.

McLeish, Don. 2011. “A General Method for Debiasing a Monte Carlo Estimator.” Monte Carlo Methods and Applications 17 (4): 301–15.

Middleton, Lawrence, George Deligiannidis, Arnaud Doucet, and Pierre E Jacob. 2019. “Unbiased Smoothing Using Particle Independent Metropolis–Hastings.” In The 22nd International Conference on Artificial Intelligence and Statistics, 2378–87. PMLR.

———. 2020. “Unbiased Markov Chain Monte Carlo for Intractable Target Distributions.” Electronic Journal of Statistics 14 (2): 2842–91.

Neal, Radford M. 1999. “Circularly-Coupled Markov Chain Sampling.” Department of Statistics, University of Toronto.

Nguyen, Tin D, Brian L Trippe, and Tamara Broderick. 2022. “Many Processors, Little Time: MCMC for Partitions via Optimal Transport Couplings.” In International Conference on Artificial Intelligence and Statistics, 3483–3514. PMLR.

O’Leary, John, and Guanyang Wang. 2024. “Metropolis–Hastings Transition Kernel Couplings.” In Annales de l’institut Henri Poincare (b) Probabilites Et Statistiques, 60:1101–24. 2. Institut Henri Poincaré.

Papp, Tamás P, and Chris Sherlock. 2024. “Scalable Couplings for the Random Walk Metropolis Algorithm.” Journal of the Royal Statistical Society Series B: Statistical Methodology, qkae113.

Rischard, Maxime, Pierre E Jacob, and Natesh Pillai. 2018. “Unbiased Estimation of Log Normalizing Constants with Applications to Bayesian Cross-Validation.” arXiv Preprint arXiv:1810.01382.

Ruiz, Francisco JR, Michalis K Titsias, Taylan Cemgil, and Arnaud Doucet. 2021. “Unbiased Gradient Estimation for Variational Auto-Encoders Using Coupled Markov Chains.” In Uncertainty in Artificial Intelligence, 707–17. PMLR.

Vanetti, Paul, and Arnaud Doucet. 2020. “Discussion on the Paper by Jacob, O’Leary, and Atchadé.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 82 (3): 584–85.

Vihola, Matti. 2018. “Unbiased Estimators and Multilevel Monte Carlo.” Operations Research 66 (2): 448–62.

Wang, Guanyang, John O’Leary, and Pierre E Jacob. 2021. “Maximal Couplings of the Metropolis–Hastings Algorithm.” In International Conference on Artificial Intelligence and Statistics, 1225–33. PMLR.

Wang, Guanyang, and Tianze Wang. 2022. “Unbiased Multilevel Monte Carlo Methods for Intractable Distributions: MLMC Meets MCMC.” arXiv Preprint arXiv:2204.04808.

Xu, Kai, Tor Erlend Fjelde, Charles Sutton, and Hong Ge. 2021. “Couplings for Multinomial Hamiltonian Monte Carlo.” In International Conference on Artificial Intelligence and Statistics, 3646–54. PMLR.

Zhu, Qiuyun, and Yves F Atchadé. 2020. “Minimax Quasi-Bayesian Estimation in Sparse Canonical Correlation Analysis via a Rayleigh Quotient Function.” arXiv Preprint arXiv:2010.08627.