Add robots.txt to exclude crawlers from walking a 'staging' test publishing dir

Review Request #2072 - Created April 13, 2015 and submitted

Information
Eric Ayers
pants
zundel/add-robots-txt
1416
b876188...
Reviewers
pants-reviews
jsirois, lahosken

Add robots.txt to exclude crawlers from walking a 'staging' test publishing dir

See https://support.google.com/webmasters/answer/6062608?hl=en&ref_topic=6061961

$ ./build-support/bin/publish_docs.sh

robots.txt is copied to root of docs

./build-support/bin/publish_docs.sh -d zundel -p

Now publishes the site under the directory to staging/zundel where it will be excluded from crawlers that obey the robots.txt settings.

John Sirois
Larry Hosken
Eric Ayers
Review request changed

Status: Closed (submitted)

Change Summary:

Thanks for the reviews John & Larry. Commit 734bd04

Loading...