Teraru.jp

SEO勉強中。アウトプットとメモ用

Wikipediaのrobotsが説明的だ・・

SEO

Wikipediaのrobots.txtが気になったので覗いてみたら、めちゃくちゃ丁寧にコメントアウトで説明書きしてあった。

 

読み解くだけで勉強になりそうなので、時間がある日に全文読み解き記事を出そう。

一旦貼り付けます。JST2024年4月6日2時34分の情報。

 

 

 

  1. # robots.txt for http://www.wikipedia.org/ and friends
  2. #
  3. # Please note: There are a lot of pages on this site, and there are
  4. # some misbehaved spiders out there that go _way_ too fast. If you're
  5. # irresponsible, your access to the site may be blocked.
  6. #
  7.  
  8. # Observed spamming large amounts of https://en.wikipedia.org/?curid=NNNNNN
  9. # and ignoring 429 ratelimit responses, claims to respect robots:
  10. # http://mj12bot.com/
  11. User-agent: MJ12bot
  12. Disallow: /
  13.  
  14. # advertising-related bots:
  15. User-agent: Mediapartners-Google*
  16. Disallow: /
  17.  
  18. # Wikipedia work bots:
  19. User-agent: IsraBot
  20. Disallow:
  21.  
  22. User-agent: Orthogaffe
  23. Disallow:
  24.  
  25. # Crawlers that are kind enough to obey, but which we'd rather not have
  26. # unless they're feeding search engines.
  27. User-agent: UbiCrawler
  28. Disallow: /
  29.  
  30. User-agent: DOC
  31. Disallow: /
  32.  
  33. User-agent: Zao
  34. Disallow: /
  35.  
  36. # Some bots are known to be trouble, particularly those designed to copy
  37. # entire sites. Please obey robots.txt.
  38. User-agent: sitecheck.internetseer.com
  39. Disallow: /
  40.  
  41. User-agent: Zealbot
  42. Disallow: /
  43.  
  44. User-agent: MSIECrawler
  45. Disallow: /
  46.  
  47. User-agent: SiteSnagger
  48. Disallow: /
  49.  
  50. User-agent: WebStripper
  51. Disallow: /
  52.  
  53. User-agent: WebCopier
  54. Disallow: /
  55.  
  56. User-agent: Fetch
  57. Disallow: /
  58.  
  59. User-agent: Offline Explorer
  60. Disallow: /
  61.  
  62. User-agent: Teleport
  63. Disallow: /
  64.  
  65. User-agent: TeleportPro
  66. Disallow: /
  67.  
  68. User-agent: WebZIP
  69. Disallow: /
  70.  
  71. User-agent: linko
  72. Disallow: /
  73.  
  74. User-agent: HTTrack
  75. Disallow: /
  76.  
  77. User-agent: Microsoft.URL.Control
  78. Disallow: /
  79.  
  80. User-agent: Xenu
  81. Disallow: /
  82.  
  83. User-agent: larbin
  84. Disallow: /
  85.  
  86. User-agent: libwww
  87. Disallow: /
  88.  
  89. User-agent: ZyBORG
  90. Disallow: /
  91.  
  92. User-agent: Download Ninja
  93. Disallow: /
  94.  
  95. # Misbehaving: requests much too fast:
  96. User-agent: fast
  97. Disallow: /
  98.  
  99. #
  100. # Sorry, wget in its recursive mode is a frequent problem.
  101. # Please read the man page and use it properly; there is a
  102. # --wait option you can use to set the delay between hits,
  103. # for instance.
  104. #
  105. User-agent: wget
  106. Disallow: /
  107.  
  108. #
  109. # The 'grub' distributed client has been *very* poorly behaved.
  110. #
  111. User-agent: grub-client
  112. Disallow: /
  113.  
  114. #
  115. # Doesn't follow robots.txt anyway, but...
  116. #
  117. User-agent: k2spider
  118. Disallow: /
  119.  
  120. #
  121. # Hits many times per second, not acceptable
  122. # http://www.nameprotect.com/botinfo.html
  123. User-agent: NPBot
  124. Disallow: /
  125.  
  126. # A capture bot, downloads gazillions of pages with no public benefit
  127. # http://www.webreaper.net/
  128. User-agent: WebReaper
  129. Disallow: /
  130.  
  131.  
  132. #
  133. # Friendly, low-speed bots are welcome viewing article pages, but not
  134. # dynamically-generated pages please.
  135. #
  136. # Inktomi's "Slurp" can read a minimum delay between hits; if your
  137. # bot supports such a thing using the 'Crawl-delay' or another
  138. # instruction, please let us know.
  139. #
  140. # There is a special exception for API mobileview to allow dynamic
  141. # mobile web & app views to load section content.
  142. # These views aren't HTTP-cached but use parser cache aggressively
  143. # and don't expose special: pages etc.
  144. #
  145. # Another exception is for REST API documentation, located at
  146. # /api/rest_v1/?doc.
  147. #
  148. User-agent: *
  149. Allow: /w/api.php?action=mobileview&
  150. Allow: /w/load.php?
  151. Allow: /api/rest_v1/?doc
  152. Disallow: /w/
  153. Disallow: /api/
  154. Disallow: /trap/
  155. Disallow: /wiki/Special:
  156. Disallow: /wiki/Spezial:
  157. Disallow: /wiki/Spesial:
  158. Disallow: /wiki/Special%3A
  159. Disallow: /wiki/Spezial%3A
  160. Disallow: /wiki/Spesial%3A
  161. #
  162. # ar:
  163. Disallow: /wiki/%D8%AE%D8%A7%D8%B5:Search
  164. Disallow: /wiki/%D8%AE%D8%A7%D8%B5%3ASearch
  165. #
  166. # dewiki:
  167. # T6937
  168. # sensible deletion and meta user discussion pages:
  169. Disallow: /wiki/Wikipedia:L%C3%B6schkandidaten/
  170. Disallow: /wiki/Wikipedia:Löschkandidaten/
  171. Disallow: /wiki/Wikipedia:Vandalensperrung/
  172. Disallow: /wiki/Wikipedia:Benutzersperrung/
  173. Disallow: /wiki/Wikipedia:Vermittlungsausschuss/
  174. Disallow: /wiki/Wikipedia:Administratoren/Probleme/
  175. Disallow: /wiki/Wikipedia:Adminkandidaturen/
  176. Disallow: /wiki/Wikipedia:Qualitätssicherung/
  177. Disallow: /wiki/Wikipedia:Qualit%C3%A4tssicherung/
  178. # 4937#5
  179. Disallow: /wiki/Wikipedia:Vandalismusmeldung/
  180. Disallow: /wiki/Wikipedia:Gesperrte_Lemmata/
  181. Disallow: /wiki/Wikipedia:Löschprüfung/
  182. Disallow: /wiki/Wikipedia:L%C3%B6schprüfung/
  183. Disallow: /wiki/Wikipedia:Administratoren/Notizen/
  184. Disallow: /wiki/Wikipedia:Schiedsgericht/Anfragen/
  185. Disallow: /wiki/Wikipedia:L%C3%B6schpr%C3%BCfung/
  186. # T14111
  187. Disallow: /wiki/Wikipedia:Checkuser/
  188. Disallow: /wiki/Wikipedia_Diskussion:Checkuser/
  189. Disallow: /wiki/Wikipedia_Diskussion:Adminkandidaturen/
  190. # T15961
  191. Disallow: /wiki/Wikipedia:Spam-Blacklist-Log
  192. Disallow: /wiki/Wikipedia%3ASpam-Blacklist-Log
  193. Disallow: /wiki/Wikipedia_Diskussion:Spam-Blacklist-Log
  194. Disallow: /wiki/Wikipedia_Diskussion%3ASpam-Blacklist-Log
  195. #
  196. # enwiki:
  197. # Folks get annoyed when VfD discussions end up the number 1 google hit for
  198. # their name. See T6776
  199. Disallow: /wiki/Wikipedia:Articles_for_deletion/
  200. Disallow: /wiki/Wikipedia%3AArticles_for_deletion/
  201. Disallow: /wiki/Wikipedia:Votes_for_deletion/
  202. Disallow: /wiki/Wikipedia%3AVotes_for_deletion/
  203. Disallow: /wiki/Wikipedia:Pages_for_deletion/
  204. Disallow: /wiki/Wikipedia%3APages_for_deletion/
  205. Disallow: /wiki/Wikipedia:Miscellany_for_deletion/
  206. Disallow: /wiki/Wikipedia%3AMiscellany_for_deletion/
  207. Disallow: /wiki/Wikipedia:Miscellaneous_deletion/
  208. Disallow: /wiki/Wikipedia%3AMiscellaneous_deletion/
  209. Disallow: /wiki/Wikipedia:Copyright_problems
  210. Disallow: /wiki/Wikipedia%3ACopyright_problems
  211. Disallow: /wiki/Wikipedia:Protected_titles/
  212. Disallow: /wiki/Wikipedia%3AProtected_titles/
  213. # T15398
  214. Disallow: /wiki/Wikipedia:WikiProject_Spam/
  215. Disallow: /wiki/Wikipedia%3AWikiProject_Spam/
  216. # T16075
  217. Disallow: /wiki/MediaWiki:Spam-blacklist
  218. Disallow: /wiki/MediaWiki%3ASpam-blacklist
  219. Disallow: /wiki/MediaWiki_talk:Spam-blacklist
  220. Disallow: /wiki/MediaWiki_talk%3ASpam-blacklist
  221. # T13261
  222. Disallow: /wiki/Wikipedia:Requests_for_arbitration/
  223. Disallow: /wiki/Wikipedia%3ARequests_for_arbitration/
  224. Disallow: /wiki/Wikipedia:Requests_for_comment/
  225. Disallow: /wiki/Wikipedia%3ARequests_for_comment/
  226. Disallow: /wiki/Wikipedia:Requests_for_adminship/
  227. Disallow: /wiki/Wikipedia%3ARequests_for_adminship/
  228. # T12288
  229. Disallow: /wiki/Wikipedia_talk:Articles_for_deletion/
  230. Disallow: /wiki/Wikipedia_talk%3AArticles_for_deletion/
  231. Disallow: /wiki/Wikipedia_talk:Votes_for_deletion/
  232. Disallow: /wiki/Wikipedia_talk%3AVotes_for_deletion/
  233. Disallow: /wiki/Wikipedia_talk:Pages_for_deletion/
  234. Disallow: /wiki/Wikipedia_talk%3APages_for_deletion/
  235. Disallow: /wiki/Wikipedia_talk:Miscellany_for_deletion/
  236. Disallow: /wiki/Wikipedia_talk%3AMiscellany_for_deletion/
  237. Disallow: /wiki/Wikipedia_talk:Miscellaneous_deletion/
  238. Disallow: /wiki/Wikipedia_talk%3AMiscellaneous_deletion/
  239. # T16793
  240. Disallow: /wiki/Wikipedia:Changing_username
  241. Disallow: /wiki/Wikipedia%3AChanging_username
  242. Disallow: /wiki/Wikipedia:Changing_username/
  243. Disallow: /wiki/Wikipedia%3AChanging_username/
  244. Disallow: /wiki/Wikipedia_talk:Changing_username
  245. Disallow: /wiki/Wikipedia_talk%3AChanging_username
  246. Disallow: /wiki/Wikipedia_talk:Changing_username/
  247. Disallow: /wiki/Wikipedia_talk%3AChanging_username/
  248. #
  249. # eswiki:
  250. # T8746
  251. Disallow: /wiki/Wikipedia:Consultas_de_borrado/
  252. Disallow: /wiki/Wikipedia%3AConsultas_de_borrado/
  253. #
  254. # fiwiki:
  255. # T10695
  256. Disallow: /wiki/Wikipedia:Poistettavat_sivut
  257. Disallow: /wiki/K%C3%A4ytt%C3%A4j%C3%A4:
  258. Disallow: /wiki/Käyttäjä:
  259. Disallow: /wiki/Keskustelu_k%C3%A4ytt%C3%A4j%C3%A4st%C3%A4:
  260. Disallow: /wiki/Keskustelu_käyttäjästä:
  261. Disallow: /wiki/Wikipedia:Yll%C3%A4pit%C3%A4j%C3%A4t/
  262. Disallow: /wiki/Wikipedia:Ylläpitäjät/
  263. #
  264. # hewiki:
  265. Disallow: /wiki/%D7%9E%D7%99%D7%95%D7%97%D7%93:Search
  266. Disallow: /wiki/%D7%9E%D7%99%D7%95%D7%97%D7%93%3ASearch
  267. #T11517
  268. Disallow: /wiki/????????:?????_???????_??????/
  269. Disallow: /wiki/????????%3A?????_???????_??????/
  270. Disallow: /wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94:%D7%A8%D7%A9%D7%99%D7%9E%D7%AA_%D7%9E%D7%95%D7%A2%D7%9E%D7%93%D7%99%D7%9D_%D7%9C%D7%9E%D7%97%D7%99%D7%A7%D7%94/
  271. Disallow: /wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94%3A%D7%A8%D7%A9%D7%99%D7%9E%D7%AA_%D7%9E%D7%95%D7%A2%D7%9E%D7%93%D7%99%D7%9D_%D7%9C%D7%9E%D7%97%D7%99%D7%A7%D7%94/
  272. Disallow: /wiki/????????:?????_??_??????_???????
  273. Disallow: /wiki/????????%3A?????_??_??????_???????
  274. Disallow: /wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94:%D7%A2%D7%A8%D7%9B%D7%99%D7%9D_%D7%9C%D7%90_%D7%A7%D7%99%D7%99%D7%9E%D7%99%D7%9D_%D7%95%D7%9E%D7%95%D7%92%D7%A0%D7%99%D7%9D
  275. Disallow: /wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94%3A%D7%A2%D7%A8%D7%9B%D7%99%D7%9D_%D7%9C%D7%90_%D7%A7%D7%99%D7%99%D7%9E%D7%99%D7%9D_%D7%95%D7%9E%D7%95%D7%92%D7%A0%D7%99%D7%9D
  276. Disallow: /wiki/????????:????_??_??????_???????
  277. Disallow: /wiki/????????%3A????_??_??????_???????
  278. Disallow: /wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94:%D7%93%D7%A4%D7%99%D7%9D_%D7%9C%D7%90_%D7%A7%D7%99%D7%99%D7%9E%D7%99%D7%9D_%D7%95%D7%9E%D7%95%D7%92%D7%A0%D7%99%D7%9D
  279. Disallow: /wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94%3A%D7%93%D7%A4%D7%99%D7%9D_%D7%9C%D7%90_%D7%A7%D7%99%D7%99%D7%9E%D7%99%D7%9D_%D7%95%D7%9E%D7%95%D7%92%D7%A0%D7%99%D7%9D
  280. #
  281. # huwiki:
  282. Disallow: /wiki/Speci%C3%A1lis:Search
  283. Disallow: /wiki/Speci%C3%A1lis%3ASearch
  284. #
  285. # itwiki:
  286. # T7545
  287. Disallow: /wiki/Wikipedia:Pagine_da_cancellare
  288. Disallow: /wiki/Wikipedia%3APagine_da_cancellare
  289. Disallow: /wiki/Wikipedia:Utenti_problematici
  290. Disallow: /wiki/Wikipedia%3AUtenti_problematici
  291. Disallow: /wiki/Wikipedia:Vandalismi_in_corso
  292. Disallow: /wiki/Wikipedia%3AVandalismi_in_corso
  293. Disallow: /wiki/Wikipedia:Amministratori
  294. Disallow: /wiki/Wikipedia%3AAmministratori
  295. Disallow: /wiki/Wikipedia:Proposte_di_cancellazione_semplificata
  296. Disallow: /wiki/Wikipedia%3AProposte_di_cancellazione_semplificata
  297. Disallow: /wiki/Categoria:Da_cancellare_subito
  298. Disallow: /wiki/Categoria%3ADa_cancellare_subito
  299. Disallow: /wiki/Wikipedia:Sospette_violazioni_di_copyright
  300. Disallow: /wiki/Wikipedia%3ASospette_violazioni_di_copyright
  301. Disallow: /wiki/Categoria:Da_controllare_per_copyright
  302. Disallow: /wiki/Categoria%3ADa_controllare_per_copyright
  303. Disallow: /wiki/Progetto:Rimozione_contributi_sospetti
  304. Disallow: /wiki/Progetto%3ARimozione_contributi_sospetti
  305. Disallow: /wiki/Categoria:Da_cancellare_subito_per_violazione_integrale_copyright
  306. Disallow: /wiki/Categoria%3ADa_cancellare_subito_per_violazione_integrale_copyright
  307. Disallow: /wiki/Progetto:Cococo
  308. Disallow: /wiki/Progetto%3ACococo
  309. Disallow: /wiki/Discussioni_progetto:Cococo
  310. Disallow: /wiki/Discussioni_progetto%3ACococo
  311. #
  312. # jawiki
  313. Disallow: /wiki/%E7%89%B9%E5%88%A5:Search
  314. Disallow: /wiki/%E7%89%B9%E5%88%A5%3ASearch
  315. # T7239
  316. Disallow: /wiki/Wikipedia:%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC/
  317. Disallow: /wiki/Wikipedia%3A%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC/
  318. Disallow: /wiki/Wikipedia:%E5%88%A9%E7%94%A8%E8%80%85%E3%83%9A%E3%83%BC%E3%82%B8%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  319. Disallow: /wiki/Wikipedia%3A%E5%88%A9%E7%94%A8%E8%80%85%E3%83%9A%E3%83%BC%E3%82%B8%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  320. # nowiki
  321. # T13432
  322. Disallow: /wiki/Bruker:
  323. Disallow: /wiki/Bruker%3A
  324. Disallow: /wiki/Brukerdiskusjon
  325. Disallow: /wiki/Wikipedia:Administratorer
  326. Disallow: /wiki/Wikipedia%3AAdministratorer
  327. Disallow: /wiki/Wikipedia-diskusjon:Administratorer
  328. Disallow: /wiki/Wikipedia-diskusjon%3AAdministratorer
  329. Disallow: /wiki/Wikipedia:Sletting
  330. Disallow: /wiki/Wikipedia%3ASletting
  331. Disallow: /wiki/Wikipedia-diskusjon:Sletting
  332. Disallow: /wiki/Wikipedia-diskusjon%3ASletting
  333. #
  334. # plwiki
  335. # T10067
  336. Disallow: /wiki/Wikipedia:Strony_do_usuni%C4%99cia
  337. Disallow: /wiki/Wikipedia%3AStrony_do_usuni%C4%99cia
  338. Disallow: /wiki/Wikipedia:Do_usuni%C4%99cia
  339. Disallow: /wiki/Wikipedia%3ADo_usuni%C4%99cia
  340. Disallow: /wiki/Wikipedia:SDU/
  341. Disallow: /wiki/Wikipedia%3ASDU/
  342. Disallow: /wiki/Wikipedia:Strony_podejrzane_o_naruszenie_praw_autorskich
  343. Disallow: /wiki/Wikipedia%3AStrony_podejrzane_o_naruszenie_praw_autorskich
  344. #
  345. # ptwiki:
  346. # T7394
  347. Disallow: /wiki/Wikipedia:Páginas_para_eliminar/
  348. Disallow: /wiki/Wikipedia:P%C3%A1ginas_para_eliminar/
  349. Disallow: /wiki/Wikipedia%3AP%C3%A1ginas_para_eliminar/
  350. Disallow: /wiki/Wikipedia_Discussão:Páginas_para_eliminar/
  351. Disallow: /wiki/Wikipedia_Discuss%C3%A3o:P%C3%A1ginas_para_eliminar/
  352. Disallow: /wiki/Wikipedia_Discuss%C3%A3o%3AP%C3%A1ginas_para_eliminar/
  353. #
  354. # rowiki:
  355. # T14546
  356. Disallow: /wiki/Wikipedia:Pagini_de_%C5%9Fters
  357. Disallow: /wiki/Wikipedia%3APagini_de_%C5%9Fters
  358. Disallow: /wiki/Discu%C5%A3ie_Wikipedia:Pagini_de_%C5%9Fters
  359. Disallow: /wiki/Discu%C5%A3ie_Wikipedia%3APagini_de_%C5%9Fters
  360. #
  361. # ruwiki:
  362. Disallow: /wiki/%D0%A1%D0%BF%D0%B5%D1%86%D0%B8%D0%B0%D0%BB%D1%8C%D0%BD%D1%8B%D0%B5:Search
  363. Disallow: /wiki/%D0%A1%D0%BF%D0%B5%D1%86%D0%B8%D0%B0%D0%BB%D1%8C%D0%BD%D1%8B%D0%B5%3ASearch
  364. #
  365. # svwiki:
  366. # T12229
  367. Disallow: /wiki/Wikipedia%3ASidor_f%C3%B6reslagna_f%C3%B6r_radering
  368. Disallow: /wiki/Wikipedia:Sidor_f%C3%B6reslagna_f%C3%B6r_radering
  369. Disallow: /wiki/Wikipedia:Sidor_föreslagna_för_radering
  370. Disallow: /wiki/Användare
  371. Disallow: /wiki/Anv%C3%A4ndare
  372. Disallow: /wiki/Användardiskussion
  373. Disallow: /wiki/Anv%C3%A4ndardiskussion
  374. Disallow: /wiki/Wikipedia:Skyddade_sidnamn
  375. Disallow: /wiki/Wikipedia%3ASkyddade_sidnamn
  376. # T13291
  377. Disallow: /wiki/Wikipedia:Sidor_som_bör_raderas
  378. Disallow: /wiki/Wikipedia:Sidor_som_b%C3%B6r_raderas
  379. Disallow: /wiki/Wikipedia%3ASidor_som_b%C3%B6r_raderas
  380. #
  381. # zhwiki:
  382. # T7104
  383. Disallow: /wiki/Wikipedia:?除投票/侵?
  384. Disallow: /wiki/Wikipedia:%E5%88%A0%E9%99%A4%E6%8A%95%E7%A5%A8/%E4%BE%B5%E6%9D%83
  385. Disallow: /wiki/Wikipedia:?除投票和?求
  386. Disallow: /wiki/Wikipedia:%E5%88%A0%E9%99%A4%E6%8A%95%E7%A5%A8%E5%92%8C%E8%AF%B7%E6%B1%82
  387. Disallow: /wiki/Category:快速?除候?
  388. Disallow: /wiki/Category:%E5%BF%AB%E9%80%9F%E5%88%A0%E9%99%A4%E5%80%99%E9%80%89
  389. Disallow: /wiki/Category:?基百科需要翻?的文章
  390. Disallow: /wiki/Category:%E7%BB%B4%E5%9F%BA%E7%99%BE%E7%A7%91%E9%9C%80%E8%A6%81%E7%BF%BB%E8%AF%91%E7%9A%84%E6%96%87%E7%AB%A0
  391. #
  392. # sister projects
  393. #
  394. # enwikinews:
  395. # T7340
  396. Disallow: /wiki/Portal:Prepared_stories/
  397. Disallow: /wiki/Portal%3APrepared_stories/
  398. #
  399. # itwikinews
  400. # T11138
  401. Disallow: /wiki/Wikinotizie:Richieste_di_cancellazione
  402. Disallow: /wiki/Wikinotizie:Sospette_violazioni_di_copyright
  403. Disallow: /wiki/Categoria:Da_cancellare_subito
  404. Disallow: /wiki/Categoria:Da_cancellare_subito_per_violazione_integrale_copyright
  405. Disallow: /wiki/Wikinotizie:Storie_in_preparazione
  406. #
  407. # enwikiquote:
  408. # T17095
  409. Disallow: /wiki/Wikiquote:Votes_for_deletion/
  410. Disallow: /wiki/Wikiquote%3AVotes_for_deletion/
  411. Disallow: /wiki/Wikiquote_talk:Votes_for_deletion/
  412. Disallow: /wiki/Wikiquote_talk%3AVotes_for_deletion/
  413. Disallow: /wiki/Wikiquote:Votes_for_deletion_archive/
  414. Disallow: /wiki/Wikiquote%3AVotes_for_deletion_archive/
  415. Disallow: /wiki/Wikiquote_talk:Votes_for_deletion_archive/
  416. Disallow: /wiki/Wikiquote_talk%3AVotes_for_deletion_archive/
  417. #
  418. # enwikibooks
  419. Disallow: /wiki/Wikibooks:Votes_for_deletion
  420. #
  421. # working...
  422. Disallow: /wiki/Fundraising_2007/comments
  423. #
  424. #
  425. #
  426. #----------------------------------------------------------#
  427. #
  428. #
  429. #
  430.  # ここに書かれた内容はグローバルRobots.txtに追加されます。設定はローカルでのみ有効です。<!--この行頭のスペースは除去しないでください。--><pre>
  431. #
  432. # ブラックリスト関係
  433. Disallow: /wiki/MediaWiki:Spam-blacklist
  434. Disallow: /wiki/MediaWiki%3ASpam-blacklist
  435. Disallow: /wiki/MediaWiki%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:Spam-blacklist
  436. Disallow: /wiki/MediaWiki%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3ASpam-blacklist
  437. Disallow: /wiki/MediaWiki:Titleblacklist
  438. Disallow: /wiki/MediaWiki%3ATitleblacklist
  439. Disallow: /wiki/MediaWiki%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:Titleblacklist
  440. Disallow: /wiki/MediaWiki%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3ATitleblacklist
  441. #
  442. # 削除依頼関係
  443. Disallow: /wiki/Wikipedia:削除依頼
  444. Disallow: /wiki/Wikipedia:%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  445. Disallow: /wiki/Wikipedia%3A%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  446. Disallow: /wiki/Wikipedia‐ノート:削除依頼
  447. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  448. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  449. Disallow: /wiki/Wikipedia:利用者ページの削除依頼
  450. Disallow: /wiki/Wikipedia:%E5%88%A9%E7%94%A8%E8%80%85%E3%83%9A%E3%83%BC%E3%82%B8%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  451. Disallow: /wiki/Wikipedia%3A%E5%88%A9%E7%94%A8%E8%80%85%E3%83%9A%E3%83%BC%E3%82%B8%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  452. Disallow: /wiki/Wikipedia‐ノート:利用者ページの削除依頼
  453. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E5%88%A9%E7%94%A8%E8%80%85%E3%83%9A%E3%83%BC%E3%82%B8%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  454. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E5%88%A9%E7%94%A8%E8%80%85%E3%83%9A%E3%83%BC%E3%82%B8%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  455. Disallow: /wiki/Wikipedia:リダイレクトの削除依頼
  456. Disallow: /wiki/Wikipedia:%E3%83%AA%E3%83%80%E3%82%A4%E3%83%AC%E3%82%AF%E3%83%88%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  457. Disallow: /wiki/Wikipedia%3A%E3%83%AA%E3%83%80%E3%82%A4%E3%83%AC%E3%82%AF%E3%83%88%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  458. Disallow: /wiki/Wikipedia‐ノート:リダイレクトの削除依頼
  459. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E3%83%AA%E3%83%80%E3%82%A4%E3%83%AC%E3%82%AF%E3%83%88%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  460. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E3%83%AA%E3%83%80%E3%82%A4%E3%83%AC%E3%82%AF%E3%83%88%E3%81%AE%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC
  461. Disallow: /wiki/Wikipedia:削除の過去ログ
  462. Disallow: /wiki/Wikipedia:%E5%89%8A%E9%99%A4%E3%81%AE%E9%81%8E%E5%8E%BB%E3%83%AD%E3%82%B0
  463. Disallow: /wiki/Wikipedia%3A%E5%89%8A%E9%99%A4%E3%81%AE%E9%81%8E%E5%8E%BB%E3%83%AD%E3%82%B0
  464. Disallow: /wiki/Wikipedia:削除記録
  465. Disallow: /wiki/Wikipedia:%E5%89%8A%E9%99%A4%E8%A8%98%E9%8C%B2
  466. Disallow: /wiki/Wikipedia%3A%E5%89%8A%E9%99%A4%E8%A8%98%E9%8C%B2
  467. Disallow: /wiki/Category:削除依頼中のページ
  468. Disallow: /wiki/Category%3A%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC%E4%B8%AD%E3%81%AE%E3%83%9A%E3%83%BC%E3%82%B8
  469. Disallow: /wiki/Category:%E5%89%8A%E9%99%A4%E4%BE%9D%E9%A0%BC%E4%B8%AD%E3%81%AE%E3%83%9A%E3%83%BC%E3%82%B8
  470. Disallow: /wiki/Category:緊急案件
  471. Disallow: /wiki/Category:%E7%B7%8A%E6%80%A5%E6%A1%88%E4%BB%B6
  472. Disallow: /wiki/Category%3A%E7%B7%8A%E6%80%A5%E6%A1%88%E4%BB%B6
  473. Disallow: /wiki/Wikipedia:削除の復帰依頼
  474. Disallow: /wiki/Wikipedia:%E5%89%8A%E9%99%A4%E3%81%AE%E5%BE%A9%E5%B8%B0%E4%BE%9D%E9%A0%BC
  475. Disallow: /wiki/Wikipedia%3A%E5%89%8A%E9%99%A4%E3%81%AE%E5%BE%A9%E5%B8%B0%E4%BE%9D%E9%A0%BC
  476. #
  477. # Wikipedia名前空間へのショートカット
  478. Disallow: /wiki/WP:
  479. Disallow: /wiki/WP%3A
  480. Disallow: /wiki/ノート:WP
  481. Disallow: /wiki/%E3%83%8E%E3%83%BC%E3%83%88:WP:
  482. Disallow: /wiki/%E3%83%8E%E3%83%BC%E3%83%88%3AWP%3A
  483. Disallow: /wiki/LTA:
  484. Disallow: /wiki/LTA%3A
  485. #
  486. # 不適切な利用者名を記載する可能性があるページ
  487. Disallow: /wiki/Wikipedia:投稿ブロック依頼
  488. Disallow: /wiki/Wikipedia:%E6%8A%95%E7%A8%BF%E3%83%96%E3%83%AD%E3%83%83%E3%82%AF%E4%BE%9D%E9%A0%BC
  489. Disallow: /wiki/Wikipedia%3A%E6%8A%95%E7%A8%BF%E3%83%96%E3%83%AD%E3%83%83%E3%82%AF%E4%BE%9D%E9%A0%BC
  490. Disallow: /wiki/Wikipedia‐ノート:投稿ブロック依頼
  491. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E6%8A%95%E7%A8%BF%E3%83%96%E3%83%AD%E3%83%83%E3%82%AF%E4%BE%9D%E9%A0%BC
  492. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E6%8A%95%E7%A8%BF%E3%83%96%E3%83%AD%E3%83%83%E3%82%AF%E4%BE%9D%E9%A0%BC
  493. Disallow: /wiki/Wikipedia:管理者伝言板
  494. Disallow: /wiki/Wikipedia:%E7%AE%A1%E7%90%86%E8%80%85%E4%BC%9D%E8%A8%80%E6%9D%BF
  495. Disallow: /wiki/Wikipedia%3A%E7%AE%A1%E7%90%86%E8%80%85%E4%BC%9D%E8%A8%80%E6%9D%BF
  496. Disallow: /wiki/Wikipedia‐ノート:管理者伝言板
  497. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E7%AE%A1%E7%90%86%E8%80%85%E4%BC%9D%E8%A8%80%E6%9D%BF
  498. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E7%AE%A1%E7%90%86%E8%80%85%E4%BC%9D%E8%A8%80%E6%9D%BF
  499. Disallow: /wiki/Wikipedia:進行中の荒らし行為
  500. Disallow: /wiki/Wikipedia:%E9%80%B2%E8%A1%8C%E4%B8%AD%E3%81%AE%E8%8D%92%E3%82%89%E3%81%97%E8%A1%8C%E7%82%BA
  501. Disallow: /wiki/Wikipedia%3A%E9%80%B2%E8%A1%8C%E4%B8%AD%E3%81%AE%E8%8D%92%E3%82%89%E3%81%97%E8%A1%8C%E7%82%BA
  502. Disallow: /wiki/Wikipedia‐ノート:進行中の荒らし行為
  503. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E9%80%B2%E8%A1%8C%E4%B8%AD%E3%81%AE%E8%8D%92%E3%82%89%E3%81%97%E8%A1%8C%E7%82%BA
  504. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E9%80%B2%E8%A1%8C%E4%B8%AD%E3%81%AE%E8%8D%92%E3%82%89%E3%81%97%E8%A1%8C%E7%82%BA
  505. Disallow: /wiki/Wikipedia:利用者名変更依頼
  506. Disallow: /wiki/Wikipedia:%E5%88%A9%E7%94%A8%E8%80%85%E5%90%8D%E5%A4%89%E6%9B%B4%E4%BE%9D%E9%A0%BC
  507. Disallow: /wiki/Wikipedia%3A%E5%88%A9%E7%94%A8%E8%80%85%E5%90%8D%E5%A4%89%E6%9B%B4%E4%BE%9D%E9%A0%BC
  508. Disallow: /wiki/Wikipedia‐ノート:利用者名変更依頼
  509. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E5%88%A9%E7%94%A8%E8%80%85%E5%90%8D%E5%A4%89%E6%9B%B4%E4%BE%9D%E9%A0%BC
  510. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E5%88%A9%E7%94%A8%E8%80%85%E5%90%8D%E5%A4%89%E6%9B%B4%E4%BE%9D%E9%A0%BC
  511. Disallow: /wiki/Wikipedia:CheckUser依頼
  512. Disallow: /wiki/Wikipedia:CheckUser%E4%BE%9D%E9%A0%BC
  513. Disallow: /wiki/Wikipedia%3ACheckUser%E4%BE%9D%E9%A0%BC
  514. Disallow: /wiki/Wikipedia‐ノート:CheckUser依頼
  515. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:CheckUser%E4%BE%9D%E9%A0%BC
  516. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3ACheckUser%E4%BE%9D%E9%A0%BC
  517. Disallow: /wiki/Wikipedia:チェックユーザー依頼
  518. Disallow: /wiki/Wikipedia:%E3%83%81%E3%82%A7%E3%83%83%E3%82%AF%E3%83%A6%E3%83%BC%E3%82%B6%E3%83%BC
  519. Disallow: /wiki/Wikipedia%3A%E3%83%81%E3%82%A7%E3%83%83%E3%82%AF%E3%83%A6%E3%83%BC%E3%82%B6%E3%83%BC
  520. Disallow: /wiki/Wikipedia‐ノート:チェックユーザー依頼
  521. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88:%E3%83%81%E3%82%A7%E3%83%83%E3%82%AF%E3%83%A6%E3%83%BC%E3%82%B6%E3%83%BC
  522. Disallow: /wiki/Wikipedia%E2%80%90%E3%83%8E%E3%83%BC%E3%83%88%3A%E3%83%81%E3%82%A7%E3%83%83%E3%82%AF%E3%83%A6%E3%83%BC%E3%82%B6%E3%83%BC
  523. #
  524. # 利用者ページ・会話ページ
  525. Disallow: /wiki/利用者:
  526. Disallow: /wiki/%E5%88%A9%E7%94%A8%E8%80%85:
  527. Disallow: /wiki/%E5%88%A9%E7%94%A8%E8%80%85%3A
  528. Disallow: /wiki/利用者‐会話:
  529. Disallow: /wiki/%E5%88%A9%E7%94%A8%E8%80%85%E2%80%90%E4%BC%9A%E8%A9%B1:
  530. Disallow: /wiki/%E5%88%A9%E7%94%A8%E8%80%85%E2%80%90%E4%BC%9A%E8%A9%B1%3A
  531. #
  532. # </pre>
  533.  

「きらいじゃないけどブロックするよ」的なこととか、「このrobotsに従ってね」とか、クローラー（並びにその開発者）に語りかけていて面白い。それに、なぜそのクローラーをブロックしているのか明示しているので、robotsの勉強としてはちょうど良さそう。