This is controlled by the ACL that is run for each incoming RCPT command. It is defined by the acl_smtp_rcpt option. You can check this part of your configuration by using the -bh option to run a simulated SMTP session, during which Exim will tell you what things it is checking.