Seminar 2024-12-04: Väinö Yrjänäinen

Abstract Quantifying uncertainty in word embeddings is crucial for reliable inference from textual data, yet existing methods like bootstrap and mean-field variational inference are computationally infeasible or make limiting assumptions. We explore alternative approaches of uncertainty quantification via posterior sampling of word embeddings. We present Gibbs sampling using Polya-Gamma augmentation, alongside Laplace approximation and Hamiltonian Monte Carlo. Additionally, we address the challenge of non-identifiability in word2vec-based word embeddings. In simulation studies with known ground truth, we show that our Gibbs sampler and Hamiltonian Monte Carlo outperform variational inference. Moreover, we use the MovieLens data set to demonstrate our methods' feasibility and accuracy on real data.