Info Bottleneck
- On the Information Bottleneck Theory of Deep Learning | OpenReview # 392
-
[1503.02406] Deep Learning and the Information Bottleneck Principle # 1068
– [physics/0004057] The information bottleneck method # 2981
Graph IB
- [2112.08903] Graph Structure Learning with Variational Information Bottleneck
- Contrastive Graph Structure Learning via Information Bottleneck for Recommendation
- GitHub - snap-stanford/GIB: Graph Information Bottleneck (GIB) for learning minimal sufficient structural and feature information using GNNs
- [2112.08903] Graph Structure Learning with Variational Information Bottleneck
Deep Learning on hi-dimensional data
- ICML 18: Mutual Information Neural Estimation # 467
– MINE, seed work on high dimensional
– explain article Explanation of Mutual Information Neural Estimation - Ruihong Qiu
– implementation https://github.com/sungyubkim/MINE-Mutual-Information-Neural-Estimation-/blob/master/MINE.ipynb
– implementation GitHub - gtegner/mine-pytorch: Mutual Information Neural Estimation in Pytorch - ICML 21: Decomposed Mutual Information Estimation for Contrastive Representation Learning # 12
- ICLR 20: Understanding the Limitations of Variational Mutual Information Estimators | OpenReview # 118
– Code of all below: GitHub - ermongroup/smile-mi-estimator: PyTorch implementation for the ICLR 2020 paper "Understanding the Limitations of Variational Mutual Information Estimators" - AISTAT 20: Formal Limitations on the Measurement of Mutual Information #168
-
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization # 1417
– f-GAN, NWJ estimator -
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization | IEEE Journals & Magazine | IEEE Xplore # 641
– NWJ estimator
– conf version Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization # 101 - Arxiv 18: [1807.03748] Representation Learning with Contrastive Predictive Coding, # 4012
– CPS estimator - ICML 19: On Variational Bounds of Mutual Information # 444
– CPS & NWJ estimator
-
Measuring and Improving the Use of Graph Information in Graph Neural Networks | OpenReview
– a baseline to measure quantity and quality of information obtained from graph data -
High-dimensional mutual information estimation for image registration | IEEE Conference Publication | IEEE Xplore
– useful for vector-matrix MI - https://journals.aps.org/pre/pdf/10.1103/PhysRevE.69.066138
- [1606.05229] Estimating mutual information in high dimensions via classification error
- NeurIPS 20 [2010.12811] Graph Information Bottleneck # 73
– under graph scenario - ICLR 19 Deep Graph Infomax | OpenReview # 1001
– under graph scenario -
Which Mutual-Information Representation Learning Objectives are Sufficient for Control?
– under RL - Efficient Estimation of Mutual Information for Strongly Dependent Variables
-
[1808.06670] Learning deep representations by mutual information estimation and maximization
– Deep InfoMax - [2101.10160] Measuring Dependence with Matrix-based Entropy Functional
- ICLR 20 [2010.01766] DEMI: Discriminative Estimator of Mutual Information, #5, rejected
– high dimensional -
[2106.14646] On Study of Mutual Information and its Estimation Methods
– tutorial