1 Topic by Dwedit (edited by Dwedit 2008-03-02 22:20)

  • Dwedit
  • Junior Member
  • Offline
  • Registered: 2007-01-18
  • Posts: 9

Topic: Dummy index.html files are missing robots meta tag

The dummy index.html files really should have this tag, so that google doesn't pick up those pages.

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

I've already seen google pick up a few dummy pages from the Attachment Mod.

  • Smartys
  • Former PunBB Developer
  • Offline
  • Registered: 2004-04-30
  • Posts: 7,844

Re: Dummy index.html files are missing robots meta tag

Not only is someone linking to blank index.html files, but Google is indexing them as well? How odd. hmm

  • Dwedit
  • Junior Member
  • Offline
  • Registered: 2007-01-18
  • Posts: 9

Re: Dummy index.html files are missing robots meta tag

Yep, do a google search for "punbb img avatars" and you'll see a few dot pages on there.

  • Smartys
  • Former PunBB Developer
  • Offline
  • Registered: 2004-04-30
  • Posts: 7,844

Re: Dummy index.html files are missing robots meta tag

Interesting (from the search results, my guess would be that at some point the index.html wasn't being served as the directory index, so Google indexed the list of files in the directory and found the index.html). smile