Behavioural and self-report measures of speech perception often do not correlate well even though they claim to measure the same concept. To help reconcile previous conflicting results, this study investigated three potential reasons for the lack in correlation: 1) mismatch of International Classification of Functioning, Disability and Health (ICF) domains between measures 2) mismatch of listening situations, and 3) low test-retest reliability of each outcome measure. To do this, 26 new hearing-aid users, aged 53-83 years with bilateral mild-to-severe sensorineural hearing loss completed the following assessment tools on two occasions at least one week apart: (1) part two of the Client Orientated Scale of Improvement (COSI), (2) the Bamford-Kowal-Bench (BKB) sentence test presented in quiet, and (3) the Quick Speech-in-Noise (QuickSIN) test. Based on correlations, the COSI questions are more likely to reflect impaired hearing than activity limitations. Some high correlations between behavioural and self-report measures emerged when situations were matched more closely. Intraclass correlation coefficients (ICCs) demonstrated good to excellent test-retest reliability for overall COSI and speech perception scores. Test-retest reliability for individual COSI categories was mixed and may have contributed to some of the low correlations between measures. The behavioural speech perception tests assess only some very specific self-reported experiences, largely unrelated to the activity limitations reported as most important by many impaired listeners. Data for some individual COSI categories lacked reliability.