How to Create a Robots.txt File On DirectAdmin / WordPress

Here is a guide on how to upload a robots.txt file on DirectAdmin control panel:

Uploading Robots.txt on DirectAdmin

Robots.txt is an important file that gives instructions to search engine crawlers on how to index and access your website. With DirectAdmin control panel, uploading your custom robots.txt file is easy to do.

Why Upload Robots.txt

By uploading a robots.txt file, you can:

• Block search engines from crawling certain pages you want to keep private.

• Prevent indexing of duplicate content.

• Reduce crawl rate to manage indexation.

• Guide bots to your xml sitemap for improved indexing.

Having a proper robots.txt optimized for your site can improve SEO and give you more control over how search engines crawl your site.

How to Upload in DirectAdmin

Follow these simple steps to upload a robots.txt file on DirectAdmin:

• Login to your DirectAdmin control panel. You might have recieved the credential on your email for the same.

• Go to File Manager.

• Click on the public_html folder. This opens your main web directory.

• Create your custom robots.txt file here.

• Double click on it and then write the following text.

User-Agent: * 
Disallow: /wp-admin/ 
Allow: /wp-admin/admin-ajax.php 

Sitemap: https://example.com/sitemap_index.xml

• Don't forget to change the example.com to your website address.

That's it! Your robots.txt file is now uploaded in the root directory of your website. It may take some time for search engines to crawl and read the updated instructions.

Verifying Your Robots.txt

To verify that your robots.txt was uploaded correctly:

• Visit yoursite.com/robots.txt directly and check for your custom rules.

• Use Google Search Console to inspect the robots.txt file Google is accessing.

• Check your site's crawl stats in GSC to confirm bot behavior aligns with your directives.

• Submit a new sitemap in GSC to expedite re-crawling.

Be sure to periodically revisit your robots.txt to keep the instructions optimized for your evolving site structure and SEO strategy. Having the right directives can greatly improve search visibility and traffic!