How linguistic expressions are contextually constrained is of vital importance to our understanding of language as a formal representational system and a vehicle of social communication. This study collected behavioral and event-related potential (ERP) data to investigate neural processing of two entity-referring spatial demonstrative expressions, this one and that one, in different contexts involving the speaker, the hearer and the referred-to object. Stimulus presentation varied distance and gaze conditions with either semantically congruent or incongruent audiovisual pairings. Behavioral responses showed that distance determined the demonstrative form only in joint gaze conditions. The ERP data for the joint gaze conditions further indicated significant congruent vs. incongruent differences in the post-stimulus window of 525–725 ms for the hearer-associated spatial context. Standardized Low Resolution Brain Electromagnetic Tomography (sLORETA) showed left temporal and bilateral parietal activations for the effect. The results provide the first neural evidence that the use of spatial demonstratives in English is obligatorily influenced by two factors: (1) shared gaze of speaker and hearer, and (2) the relative distance of the object to the speaker and hearer. These findings have important implications for cognitive-linguistic theories and studies on language development and social discourse.