Bing Ignoring Instructions Not to Index Pages

by Gary on August 9, 2009 · 6 comments

in SEO

Bing has been having problems with its indexing. Just recently Brett Yount, Program Manager of Bing Webmaster Center has responded to some complaints of incidences where MSNbot disregarded specific instructions not to index certain pages.

According to mckenzie MSNbot did not follow the Disallow instruction in his robots.txt file. Another user, jiosis, complained that MSNBot ignored the robots tag where he used the NoIndex/NoFollow attribute.

The good news is that Brett Yount says that they are already working on the problem although as of the moment it will be a good idea for everyone to start checking whether pages you want to stay hidden are being indexed by Bing or not. If you encounter the same problem there is no need to fill out a content removal request since, as Brett Yount says, it is a problem on Bing’s side. All you need to do is email Brett Yount at with your domain name and “MSNBot ignoring robots tags” in the subject line and send an explanation of your problem and any proof (if any) that MSNBot is indeed the one causing the problem. Note that even without any proof (if understood it correctly) they will still look into your problem, however, for expediency’s sake it would be better to provide them with helpful details such as clips of your log file.

As for getting Bing to not index pages using the hash #, I have yet to find a complaint.

And if you are having the opposite problem, which is puzzling over why your site isn’t being indexed by Bing, don’t worry because you are definitely not alone. Bing also has some kinks to iron out in that regard. Head to the Bing community forum to read the numerous complaints and solutions on the indexing issue.

Share This Post

Scott August 18, 2009 at 4:40 pm

Hopefully bing solves these problems soon.


CoolestGeek August 28, 2009 at 4:58 am

Great tip, Hannah. Thanks for sharing.

Spunky Jones September 1, 2009 at 7:15 am

I am glad that they are working on that issue of their bot not following the instructions in the robots.txt file. However, I do like the search engine results that I am getting for one of my web directories. In fact, Bing is delivering more traffic to it that Google.

Eugenijus September 8, 2009 at 3:14 pm


David Smythe - Firstfound October 20, 2009 at 3:51 pm

I don’t like this one bit, ignoring such basic rules seems like a huge oversight on their part and potentially embarressing for companies and individuals.

Mickey W. December 8, 2009 at 12:02 pm

Bing is still young project and I think that in few years it will work without any special failures. Google also needed some time to achieve its position and we all know that it is not easy to stay on top of this branch.