Joo, W., Lee, W., Park, S., & Moon, I. C. (2020). Dirichlet Variational Autoencoder. Pattern Recognition, 107, 1-37

Abstract

This paper proposes Dirichlet Variational Autoencoder (DirVAE) using a Dirichlet prior. To infer the parameters of DirVAE, we utilize the stochastic gradient method by approximating the inverse cumulative distribution function of the Gamma distribution, which is a component of the Dirichlet distribution. This approximation on a new prior led an investigation on the component collapsing, and DirVAE revealed that the component collapsing originates from two problem sources: decoder weight collapsing and latent value collapsing. The experimental results show that 1) DirVAE generates the result with the best log-likelihood compared to the baselines; 2) DirVAE produces more interpretable latent values with no collapsing issues which the baselines suffer from; 3) the latent representation from DirVAE achieves the best classification accuracy in the (semi-)supervised classification tasks on MNIST, OMNIGLOT, COIL-20, SVHN, and CIFAR-10 compared to the baseline VAEs; and 4) the DirVAE augmented topic models show better performances in most cases. 

@article{joo-2020,

author = {Joo, Weonyoung and Lee, Wonsung and Park, Sungrae and Moon, Il‐Chul},

journal = {Pattern Recognition},

month = {11},

pages = {107514},

title = {{Dirichlet Variational Autoencoder}},

volume = {107},

year = {2020},

doi = {10.1016/j.patcog.2020.107514},

url = {https://doi.org/10.1016/j.patcog.2020.107514}

}

Source Website:

https://www.sciencedirect.com/science/article/pii/S0031320320303174