Info Bottleneck
 On the Information Bottleneck Theory of Deep Learning  OpenReview # 392

[1503.02406] Deep Learning and the Information Bottleneck Principle # 1068
– [physics/0004057] The information bottleneck method # 2981
Graph IB
 [2112.08903] Graph Structure Learning with Variational Information Bottleneck
 Contrastive Graph Structure Learning via Information Bottleneck for Recommendation
 GitHub  snapstanford/GIB: Graph Information Bottleneck (GIB) for learning minimal sufficient structural and feature information using GNNs
 [2112.08903] Graph Structure Learning with Variational Information Bottleneck
Deep Learning on hidimensional data
 ICML 18: Mutual Information Neural Estimation # 467
– MINE, seed work on high dimensional
– explain article Explanation of Mutual Information Neural Estimation  Ruihong Qiu
– implementation https://github.com/sungyubkim/MINEMutualInformationNeuralEstimation/blob/master/MINE.ipynb
– implementation GitHub  gtegner/minepytorch: Mutual Information Neural Estimation in Pytorch  ICML 21: Decomposed Mutual Information Estimation for Contrastive Representation Learning # 12
 ICLR 20: Understanding the Limitations of Variational Mutual Information Estimators  OpenReview # 118
– Code of all below: GitHub  ermongroup/smilemiestimator: PyTorch implementation for the ICLR 2020 paper "Understanding the Limitations of Variational Mutual Information Estimators"  AISTAT 20: Formal Limitations on the Measurement of Mutual Information #168

fGAN: Training Generative Neural Samplers using Variational Divergence Minimization # 1417
– fGAN, NWJ estimator 
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization  IEEE Journals & Magazine  IEEE Xplore # 641
– NWJ estimator
– conf version Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization # 101  Arxiv 18: [1807.03748] Representation Learning with Contrastive Predictive Coding, # 4012
– CPS estimator  ICML 19: On Variational Bounds of Mutual Information # 444
– CPS & NWJ estimator

Measuring and Improving the Use of Graph Information in Graph Neural Networks  OpenReview
– a baseline to measure quantity and quality of information obtained from graph data 
Highdimensional mutual information estimation for image registration  IEEE Conference Publication  IEEE Xplore
– useful for vectormatrix MI  https://journals.aps.org/pre/pdf/10.1103/PhysRevE.69.066138
 [1606.05229] Estimating mutual information in high dimensions via classification error
 NeurIPS 20 [2010.12811] Graph Information Bottleneck # 73
– under graph scenario  ICLR 19 Deep Graph Infomax  OpenReview # 1001
– under graph scenario 
Which MutualInformation Representation Learning Objectives are Sufficient for Control?
– under RL  Efficient Estimation of Mutual Information for Strongly Dependent Variables

[1808.06670] Learning deep representations by mutual information estimation and maximization
– Deep InfoMax  [2101.10160] Measuring Dependence with Matrixbased Entropy Functional
 ICLR 20 [2010.01766] DEMI: Discriminative Estimator of Mutual Information, #5, rejected
– high dimensional 
[2106.14646] On Study of Mutual Information and its Estimation Methods
– tutorial