# # robots.txt for the MOTIF web site # # Main MOTIF site : http://motif.lsce.ipsl.fr/ # # robots.txt,v 1.2 2007/07/13 13:09:27 jypeter Exp # # JYP / LSCE # # Notes : # - this file MUST be named 'robots.txt' (all lowercase), # and MUST be located in DocumentRoot # - comments start with '#' :) # - blank lines are used only to separate groups of records # (i.e. one 'User-agent:' line followed by one # or more 'Disallow:' lines # # For more details, check http://www.robotstxt.org/wc/robots.html # # Default rule for all robots # The robots are allowed to index/follow all the pages, except # when specified otherwise with a "robots directive" in a web page User-agent: * Disallow: /admin/ Disallow: /database/access/localspecs/ Disallow: /workpackages/wp1/ Disallow: /workpackages/wp2/ Disallow: /workpackages/wp3/ Disallow: /workpackages/wp4/ Disallow: /workpackages/wp5/ # The end