Usual autoencoders have an input, a hidden layer (latent space) and an output. Usual autoencoders have a latent space **smaller** than the input / output so that information has to be compressed and represented by something smaller than the input. Sparse autoencoders can have a latent space **larger** than the input dimension. But the “squeezing” of the information comes from extra regularization parameters which penalise having “active” neurons in the hidden space. This allows you to create an arbitrarily large feature set in the hidden space and still have each node correspond to a feature since only a few of them will be used due to the regularisation parameter. ## Comments <script src="https://publish-01.obsidian.md/access/0c2eeec9efe47e0b80adbe76f10aea80/publish.js"></script>