The current gold-standard for the diagnosis of hearing loss is pure-tone audiometry. Yet, the artificial pure tones used to assess hearing thresholds in pure-tone audiometry do not resemble real-life listening situations. Therefore pure-tone audiometry only provides an incomplete picture of individual hearing impairment, as disorders such as supra-threshold hearing loss (i.e. hidden hearing loss) can not be captured. Additionally, pure-tone audiometry is vastly dependent on subjective feedback. This can be problematic as giving informed feedback is challenging for some patient groups (e.g. babies that are born deaf or elderly people with dementia). Here we propose the “Neurogram”, a possible way to overcome the shortcomings of pure-tone audiometry by using a combination of system identification approaches, magnetoencephalography and a naturalistic listening situation (a radio play). By subsequently fitting linear encoding and decoding models we regress features of an acoustic signal (e.g. spectrograms) from related measured brain activity. We find that the decodability of acoustic information decreases with individual hearing capacity/impairment measured using pure-tone audiometry. Furthermore, we found a stronger relationship between subjective reports of speech perception (assessed using the Speech, Spatial and Qualities of Hearing Scale) and the here proposed “Neurogram” compared to pure-tone audiometry. In the future we aim to further develop this approach and work towards a diagnostic procedure that allows clinicians to fit hearing aids optimally based on a characterization of individual hearing impairment without solely relying on subjective feedback.