The characteristic polynomial of the GUE is at the heart of a conjecture from Fyodorov and Simm, where it is crucial to understand the log-correlated structure of the field induced. As a first step in this direction, we obtain a central limit theorem for the logarithm of the characteristic polynomial of the Gaussian $\beta$ Ensemble. Relying on the tridiagonal representation of such matrix models, we will explain how the second order recursion satisfied by the characteristic polynomial allows us to give a martingale representation of its logarithm, leading to an analysis of its fluctuations. This is a joint work with R. Butez and O. Zeitouni.