System → Administrators → {administrator} → Restrict login to trusted hosts
Specify trusted hosts to all administrators. In that case FortiGate display login form only to those IPs.
VPN > SSL-VPN Settings > Restrict Access → Limit access to specific host