SER 1 ip (1.2.3.4)

Asterisk 2 ip’s same nic as aliases.. same subnet as ser..  (1.2.3.5,1.2.3.6)

If I call asterisk from  SER and asterisk has 2 ips

When I call asterisk .

SER (1.2.3.4 ) INVITE to Asterisk (1.2.3.5)

Asterisk answer (100 Trying ) from another ip

Asterisk back (100 Trying ) to SER from (1.2.3.6) to (1.2.3.4)

Anyway to fix ?