Masters
could I get right robots.txt for CMSimple XH
Exuse me, stupid one. I wish to have correct one.
robots.txt
robots.txt
Aleksei
Re: robots.txt
There is no generally valid robots.txt.
In the robots.txt file you can specify which behavior of search robots you want on your site.
Note: It is not possible with this technique to protect web pages from access by robots or persons. You can only control the appearance in the search results.
Note: There is no guarantee that search engines will adhere to the prohibitions in robots.txt. The vast majority of robots in modern search engines take into account the presence of a robots.txt, read it and follow the instructions. Robots that search the web with bad intentions are unlikely to follow the instructions.