One Way to produce a robots.txt file is to go to the robots.txt generator page. On that page, you'll be able to set the instructions you can provide the Net crawler. You've got use of so many no cost robots.txt generator tools online, a lot of that are free of https://seotoolstube.com/