Using the decay along the diagonal of the matrix representing the perturbation with respect to the Hermite basis, we prove a reducibility result in $L^2(\mathbb{R})$ for the one-dimensional quantum harmonic oscillator perturbed by time quasi-periodic potential, via a KAM iteration. The potential is only bounded (no decay at infinity is required) and its derivative with respect to the spatial variable $x$ is allowed to grow at most like $|x|^\delta$ when $x$ goes to infinity, where the power $\delta