Google indexes MediaWiki page with url of Random page

From FVue
Jump to: navigation, search

Problem

A MediaWiki page shows up in Google with a link to Special:Random. When this link is activated, another (random) page will be presented; not the page Google indexed.

So for example, random pages of wikipedia.org are falsely indexed. Search in Google for: 

allinanchor:"Special:Random" site:wikipedia.org

Click on a link '../wiki/Special:Random'. Google will serve you another (random) page, NOT the page as suggested in the search results.

Solution

Tell search engines not to index random pages at all. Specify in robots.txt: 

User-agent: *
Disallow: /w/*
Disallow: /wiki/Special:Search
Disallow: /wiki/Special%3ASearch
Disallow: /wiki/Special:Random
Disallow: /wiki/Special%3ARandom

NOTE: Special:Randompage also works in MediaWiki, but since Special:Random is a prefix of Special:Randompage, specifying Special:Random in robots.txt suffices.

See also

3-letter words not indexed in MediaWiki: Fix MySQL to index 3-letter words of MediaWiki.

MediaWiki backup via SSH tunnel: Solution for creating a back-up of your MediaWiki if your Internet Service Provicer doesn't allow you to mysqldump

Category: MediaWiki: More MediaWiki solutions.

Journal

20060728

Installing Mediawiki - Devpit: Useful page about modifying a MediaWiki installation. Contains an example robots.txt.

http://en.wikipedia.org/robots.txt: The current robots.txt file used by Wikipedia.

20061020

This problem seems to occur in the official wikipedia.org as well, since they have in robots.txt: 

Disallow: /wiki/Special:Randompage
Disallow: /wiki/Special%3ARandompage
Disallow: /wiki/Speciaal:Randompage
Disallow: /wiki/Speciaal%3ARandompage

Mailed problem/solution to gmane.science.linguistics.wikipedia.technical. Not placed :-(?

Mailed problem/solution to gmane.org.wikimedia.mediawiki. Not placed :-(?

20061021

Mailed problem/solution to gmane.science.linguistics.wikipedia.technical. Not placed :-(?

20061022

Posted problem/solution to http://www.mwusers.com/forums/showthread.php?t=2177

20061024

Mailed problem/solution to mediawiki-l@Wikimedia.org. Placed :-)

20061031

Reported bug to bugzilla.mediawiki.org: http://bugzilla.wikipedia.org/show_bug.cgi?id=7775

Comments

blog comments powered by Disqus

Retrieved from "https://fvue.nl/w/index.php?title=Google_indexes_MediaWiki_page_with_url_of_Random_page&oldid=9368"