To communicate in difficult listening situations, individuals with hearing impairment must often invest substantial listening effort. In addition to the acoustic conditions, the degree of listening effort invested is affected by the social context of the situation. For some people, social context may increase the success importance of listening, thereby increasing their effort investment. However, for others, social context may lead to stress, particularly when there are communication breakdowns or misunderstandings. To manipulate social context in this study, we investigated the impact of social observation during a speech-in-noise task. We measured participants’ pupil and cardiovascular responses. We anticipated that social observation would increase effort investment, reflecting in higher physiological reactivity and subjective effort ratings when observed, compared to when performing the task alone. Twenty-nine experienced hearing aid users were recruited (mean age = 65 years). Participants performed the Danish Hearing in Noise Test (HINT) alone and in the presence of two observers. HINT sentences were presented at two signal-to-noise ratios (SNRs), individually adapted to target 50 and 80% of sentences correct. Baseline pupil size, peak pupil dilation, mean pupil dilation and cardiovascular parameters (heart rate variability, pre-ejection period, blood pressure and heart rate) were measured. After each block, participants rated their subjective effort investment, stress, tendency to give up and their preference to change the situation to improve audibility. Social observation increased baseline pupil size and blood pressure measures, suggesting an increase in stress. Contrary to expectations, self-report, peak pupil dilation, mean pupil dilation and the other cardiovascular measures revealed no effect of observation. Performance was also unaffected by observation. Instead, performance, self-reported effort and stress, peak pupil dilation and mean pupil dilation were sensitive to the SNR, such that performance was higher at the easier SNR, whereas self-reported effort and stress, peak and mean pupil dilation were higher at the harder SNR. These results indicate an increase in effort investment at the harder SNR compared to the easier SNR. No cardiovascular measures were sensitive to SNR. Performing a speech-in-noise task while under observation resulted in an increase in stress, rather than an increase in effort investment. Interestingly, self-reported stress did not increase under observation, but instead varied as a function of SNR, perhaps revealing that the self-report and physiological parameters measured different constructs. This study emphasizes the need for the development of listening tests that better reflect real life communication scenarios, including social context.