Quantisaton is a technique that allows for faster inference at a lower cost. That does come with a reduced accuracy of the model though. In this project, I want to analyse the impact of repeated (and more aggresive) quantisation using sparse autoencoder (SAE) analysis, to check what features degrade the fastest, and which ones are the most persistent when aggresively quantised.
(This is a project that 1. Would take me a lot of time 2. Is difficult 3. Expensive. I would love to work on it in the future)