For years we were in the top of the results when searched for webontwerp (webdesign in Dutch) in Google. No problem, none, with any update. Not in Bourbon, nore Jagger123 and not even in Bigdaddy. Until yesterday that is.
A few weeks ago we only were surprised when we discovered requests in our access_log for documents in the directory 'webontwerp' folowed by a trailing double slash. Googlebot requested documents as for example: vision2form.nl / webontwerp // documentname.html.
Normally speaking, no problem, technically a request for a document where the directory name is followed by a slash or even a double slash, is the same.
Would that just be the truth, it simply wasn't for googlebot. Did it already have problems with a 301 redirect permanent and this particular problem years ago, it seems that Bigdaddy has brought it back. That, or at least the new structure and new bot happen to have triggert this problem by trying to solve to many problems at once.
Even if it is technically speaking the same destination where to look for a document on a server, Googlebot thinks its not. It sees them as two different locations and acts on that. Meaning for our listing in the results that it disappeard when searched for 'webontwerp'.
Searching for 'site:vision2form.nl/webontwerp/' it did turn up with a URl with the mentioned double slash "//"
Due to the fact that we hadn't changed a thing and this glitch appeared over night, it had to be a problem with the googlebot. Since Google has many problems for months now, we had to react on that and find a solution our selves.
As we did have some experience with another 'canonical' problem the bot had (www. or no www. in the URl) we knew the solution had to be found in a rewriterule in the .htaccess and preferably one with a 301 redirect permanent. Not only for a trailing slash on a directory, but also for a double slash in the middle of a URL. This way all requests for a document with a double slash on the end or in the middle, would be redirected to the proper location.
Having not much or simply no experience with expressions in rewriterules, a good friend that does comes in handy. A hoster in Germany knowing his way arround Apache servers and the problems that could rise, we asked him: Pennywise - Drock.de to help us. By the way he is a briliant hoster in Germany and if you ever need to host a website there, we strongly advice you to contact him! unfortunately, by helping others, he still hasn't found the time to build his own website for his hosting services, but on his homepage you will find the details to get in contact with him.
This solution does only work on an Apache server!! And for your specific situation, we strongly advice you to get in contact with a professional!
In the .htaccess we wrote the following rules:
RewriteEngine On
RewriteCond %{REQUEST_URI} //
RewriteRule ([^/]*)/+(.*) http://vision2form.nl/$1/$2 [R=301,L]
Please note we have non-www version!
Today (may 23.) we did take a quick search in Google, to see if we did the wright thing and the bot is accepting the rules. Fortunatly it does and our page is back in the results again.
Penny, thank you!
We are able to assist and advice you in optimizing your website and search engine ranking.
Feel free to contact us! info@vision2form.nl