robots.txt file not updating on WP Engine hosted site

If you’re on WP Engine and you’re trying to modify robots.txt on a URL (i.e. before you launch the site for real), then be aware that robots.txt will always be set to ‘disallow all’ by WP Engine until you associate a custom domain with the site.

As per their website:

NOTE: By default, WP Engine restricts the traffic of search engines to any site using the domain. This means search engines will not be able to visit sites which are not currently in production using a custom domain.

Robots.txt and WP Engine, 8 May 2020