The reason behind the "We're sorry..." message

Monday, July 9, 2007 11:54 AM




Some of you might have seen this message while searching on Google, and wondered what the reason behind it might be. Instead of search results, Google displays the "We're sorry" message when we detect anomalous queries from your network. As a regular user, it is possible to answer a CAPTCHA - a reverse Turing test meant to establish that we are talking to a human user - and to continue searching. However, automated processes such as worms would have a much harder time solving the CAPTCHA. Several things can trigger the sorry message. Often it's due to infected computers or DSL routers that proxy search traffic through your network - this may be at home or even at a workplace where one or more computers might be infected. Overly aggressive SEO ranking tools may trigger this message, too. In other cases, we have seen self-propagating worms that use Google search to identify vulnerable web servers on the Internet and then exploit them. The exploited systems in turn then search Google for more vulnerable web servers and so on.  This can lead to a noticeable increase in search queries and sorry is one of our mechanisms to deal with this.

At ACM WORM 2006, we published a paper on Search Worms [PDF] that takes a much closer look at this phenomenon. Santy, one of the search worms we analyzed, looks for remote-execution vulnerabilities in the popular phpBB2 web application. In addition to exhibiting worm like propagation patterns, Santy also installs a botnet client as a payload that connects the compromised web server to an IRC channel. Adversaries can then remotely control the compromised web servers and use them for DDoS attacks, spam or phishing. Over time, the adversaries have realized that even though a botnet consisting of web servers provides a lot of aggregate bandwidth, they can increase leverage by changing the content on the compromised web servers to infect visitors and in turn join the computers of compromised visitors into much larger botnets. This fundamental change from remote attack to client based download of malware formed the basis of the research presented in our first post. In retrospect, it is interesting to see how two seemingly unrelated problems are tightly connected.
The comments you read here belong only to the person who posted them. We do, however, reserve the right to remove off-topic comments.

72 comments:

www.karmicnirvana.com said...
This comment has been removed by the author.
ibneko said...

Hmm. Maybe you guys should switch over to the reCaptcha (http://recaptcha.net/) system instead? So something productive gets done when people enter the captcha?

www.karmicnirvana.com said...
This comment has been removed by the author.
Jeremy said...

What would you have to search for to receive the error message?

free ps3 / free wii said...

I never got that error yet, but interesting to know why if I ever do get it.

J├╝rgen R. said...

I got this error once after I - as I interpreted it - searched "too fast". I am working an a Linux box behind a broadband router and I am not aware of any worm or other maleware. Is it possible to trigger this captcha by hand?

Bobly said...

It's happened to me a couple of times when using the froogle service (probably because its amount of uses before a CAPTCHA is lower than the usual search feature), though the problem for me was due to my ISP putting everyone in an area through an invisible proxy.

IWBO said...

We often see this from where I work because we have thousands of computers behind a proxy server. I'd be grateful if someone from Google could contact me to discuss how we can avoid this happening.

Jack Alexander said...

Could it be because I am using OpenDNS? I run 4 anti-virus, and 4 anti-spyware progs. and i can pretty much guarantee that this machine isn't infected. What say Google?

webado said...

All nice theories, but my case won't be explained by any of those.

I'm falling victim of a false-positive result obviously.

I'm not using a proxy, have fixed IP, work on a normal XP machine using IE7, no aggressive firewall on my part, functinal security and privacy settings meaning I accept all cookies, no browser helpers and plugins added, I'm alone on my tiny home 3-computer home network surfing Google groups or other Google services when this happens.

I have some half-baked theories involving the perennial cache pehonmenon we see at least in Groups, and maybe crossed sessions due to high overall activity (not mine), but I certainly can't put my finger on it.

codeman38 said...

Not sure if this has been fixed yet, but the last time I ended up getting the page in question, there was no option for an audio CAPTCHA. Could this be added for accessibility purposes?

in_peggy said...

I get message after 20 pages I look at. What's up with that?

Steve said...

I'm a Systems Administrator responsible for a cluster of proxy servers that provide service for about 15,000 users, and we're intermittently seeing the "Sorry" message affecting all our users.

Some advice on how to configure a legitimate proxy server to prevent this would be most welcome.

Thunderclaw said...

I must say, I'm not a techie, but I do know that I am no longer using Google as my search engine of choice as a result of these annoying captchas.

As many searches as I do in a day, I don't have the time to spend on steps that hinder my productivity when searching the web.

I really hate to say this, as I have been a huge proponent of Google since I first found out about it. Sorry Google, but this is goodbye.

art said...

What happens if this CAPTCHA screen gets no response? Presumably the IP is barred from accessing all Google sites?

I've been unable to access any Google sites from my home (cable access) PC for some weeks now, and all I can think is that the CAPTCHA page isn't reaching me.

It's not malware related as far as I can see.

Kevin

Mart said...

IF you have other search engines installed in de little search box at the right top hand of ie7, Google may receive multiple queries.
Delete all accept google .com and the error will be gone. You might have to run winsockfix or winsockxpfix to get the connection working again.
If not, and also if win update does not work, use wurtbeta
Kind regards,
Mart.

Think About It said...

While I can certainly understand the desire to minimize "bot" searches, We're being hammered by this, and the only reason I can figure is that we're being a massive proxy farm that serves many pages, not only for our school (of about 1000 users) but also for other schools in the area.

If you're going to do something like this, please have there be a way for hostmasters to contact you for to have exceptions added in to the system.

I can imaging that our proxy is hitting google at least a couple times a second.

So please, Google... How Can I contact you to get this sorted out? It's driving a lot of our students away from you.

Vadim said...
This comment has been removed by a blog administrator.
Jacques said...

We are now seeing the same issue.

We have about 2500 users behind our corporate firewalls and we do not use a proxy. This is not an issue we have ever experienced and nothing has changed as far as our internet connection.

We monitor all internet traffic and there is nothing abnormal happening from our network.

Unfortunately we have been unable to contact anyone at Google to get this resolved. We have now blocked Google completely by redirecting all Google requests to MSN. Even now after 5 hours of no-one being able to access Google except me and one of my technicians the error page is displayed every time Google is opened.

I just think it a shame that Google could get something so wrong yet am so unaccountable. Maybe the press needs to be involved to ensure that this is given the attention it requires.

Jonathan said...

Our organization has around 7,000 clients connecting through 2 squid proxy servers. Never had a problem until today many clients reported receiving the captcha message during Google searches. We are looking for malicious traffic from our end, but haven't found any yet. I wish Google could share what 'anomalous queries' from our network they have detected. It would aid our search.

myord said...

I get this all the time and I deleted all other search engines and did all I can do: virus check, spy ware etc. It's still the same.
That's why I believe it is some battle between Google and the competition where basically there are no rules. Seems Google is having hard time...

myord said...

In other words: Welcome to the No.2 Yahoo...

Ricky B said...

I get these on a home network running a fresh Ubuntu install, mostly when googling to reinstall firefox extensions. It's very annoying. Can I suggest as an improvement not asking again for a while after we verify we're not robots? It gets even more annoying with repetition.

Fish said...

"It gets even more annoying with repetition."

Boy is THAT an understatement!

This only started relatively recently with me (few days ago maybe), but seeing how I'm a "Power Googler", doing many, many searches one after the other (each with minor variations of keywords, spelling, etc) in order to find the highly technical information I'm looking for, these frequent "We're Sorry..." responses are beginning to GET ON MY NERVES! >:(

I sure wish Google would FIX THEIR LOGIC so ordinary HUMAN BEINGS (and not robots!) like me and others can do their searches without this EXTREMELY ANNOYING web page popping up all the time! Sheesh! Enough already.

-- Fish
(David B. Trout)

sanskrit87 said...

Since Google has not seen fit to even answer any of the comments here for the last 6+ months, it is probably useless to post anything at all here. But being optimistic, I will try.

This "Sorry you're a bot" thing is pretty stale by now. I understand that many less savy users have been wasting lots of hours scanning their machines. My machines are clean. i do relatively infrequent search requests. but last February and again today, I am being declared a bot for no good reason.

The explanation provided both here and on the "Sorry your a bot" page is at best inadequate and certainly misleading. For Google to do security by obscurity, is wasting legitimate user's time in the hopes of "catching" a bot???

Given the periodicity of these posts and that of my getting mis-identified as a bot, it appears that Google changes their tactics on occasion.

I really suspect that Google has perhaps something else in mind with all of this. And that is is definitely related to information gathering. Certainly the session cookies enable individual tracking until they are removed, as they do not disappear by them selves.

Very disappointed & concerned.

DailyFrame.com said...

Folks - this drives me bonkers, and I will have to evaluate whether to keep with Google going forward if it keeps up. I live by research and internet searches. I use no bots or auto stuff and there is zero spyware or virus on my PC's. All of a sudden today, I start getting these annoying things upon every new search session. If this keeps up, I'll have no other option but to get info in other places.

Jesse said...

It is so damn annoying getting these captchas. I've started using ASK.com instead of Google, and will likely change my start page if you folks don't figure out a better way of doing this.

Ordinary Coloradan said...

I am getting this on damned near ALL of my request using the search box on Firefox.

I run Kaspersky AND McAffee on my Winbox, and its locked down tight behind an OpenBSD based firewall.

This also affects my Linux (Ubuntu) machine and Firefox as well.

So its not that I got pwned. Its that your coders are idiots in that they erred and created false positives far above an allowable level.


I cleared cookies, I cleared the cache, and your idiot engine STILL throws a captcha up at EVERY damned request I submit through my search box on Firefox.

Google FIX THE FALSE POSITIVE CRAP, or start losing eyes and Yahoo here I come.

FIX IT.

SD22 said...

Totally tired of this! I am so close to switching my search engine preference. I have done all the things suggested to rid me of this problem, but not one seems to work.

To think I thought Google was so good.

Tones said...

Google is my homepage, so when I got this on my home PC I freaked out and systematically went through the process of checking for spyware and viruses etc. I also, deliberately avoided entering in the CAPTCHA just in case...

Guys, you might want to consider re-designing the page so it looks like more of an authentic error message? The Google logo, for instance looks very strange and the download links left me suspicious. Just some food for thought.

Joseph N. Hall said...

Apparently like everyone else with a proxy server that covers thousands of users, we experience "We're Sorry" messages both with and without CAPTCHAs every month or two. Given that you have so much other cool technology, this scores about 11.5 on the 1 to 10 scale of lameness.

Also annoying is when it shuts down Google Maps, and in order to get it working again, on a per-computer basis, you have to know to load an individual map tile so that you can answer the CAPTCHA for it.

I don't know what you need to do, but can you please try to address this for corporate and educational proxies? It's just pathetic.

L said...

I work at a library which has to use internet filtering. Therefore all our internet activity goes through a proxy server. This seems to be triggering the Google error on an increasingly frequent basis.

This is becoming a major concern for us and yet Google's response has been "learn to live with it because we aren't making any changes."

I find the comments saying that people are considering changing their search preferences to other engines very interesting. We may be forced to do this as well.

Jesse said...

Our small business uses a proxy server, and this has finally gotten so bad that we've started telling our users to go to ask.com instead. We haven't put in a redirect yet, but at this point it's a very likely end. At home my wife and I both have stopped using google for searches as well. There comes a point at which you're alienating valid users for the sake of a little extra security, and that is something that, in the end, will cost you far more than it gains.

Ron said...
This comment has been removed by a blog administrator.
MalPignus said...

Google Groups has tossed me around like a ragdoll for the past couple days. It keeps sending me to the sorry page without any CAPTCHA. The lockout is for Groups alone, as I can still access other Google services. Each time I get "sorry", I must sign-out, clear cookies, sign-in and access the groups page again in order to get a CAPTCHA.

I believe that Groups uses "sorry" in place of a file download cap. When I try to access files within Groups too quickly, it will lock me out, regardless of my actual queries.

If my theory is correct, the download cap should have a separate page in order to not mislead users.

If my theory is incorrect, then the bot detection contains serious flaws of logic. Not only do I get "sorry" with no CAPTCHA, but even when I do finally get a CAPTCHA, it often unlocks me only to view a single page and then locks me out again.

wh said...

our office, which uses a single proxy for about 100-150 employees, started getting these "Sorry"-Captchas today. We didn't see this last week, and our job is internet searches.

so google, please increase the tolerance, to lower false-bot-positives.

it's mainly advanced users getting annoyed by this, and you cannot want advanced users to move to yahoo, etc.

Khalid said...

could any body fix this I am just sick of it all hahah even you want to post comment here you have to have to enter the character below what irony

docsharp01 said...
This comment has been removed by a blog administrator.
MusicAndArtsBlog said...

PLEASE FIX THIS GOOGLE any office through a proxy gets burned from this and brings down havock on all that have come to rely on google, please you have us like a puppet on a string

MusicAndArtsBlog said...

please everyone when you can MOVE TO YAHOO, it was my first love and im coming back WHEN YOU CAN: YAHOO :

MusicAndArtsBlog said...
This comment has been removed by the author.
MusicAndArtsBlog said...

This "Sorry you're a bot" thing is pretty stale by now. I understand that many less savy users have been wasting lots of hours scanning their machines. My machines are clean. i do relatively infrequent search requests. but last February and again today, I am being declared a bot for no good reason.

gotta quote this, so eloquent and genius, read his entire homment DON'T BE EVIL

Stephen said...

Same as above. Using a proxy, 100-200 people and we are getting endless problems with this. Why is there nothing from google on this?

modlowarvai said...

so how do we resolve this sorry issue as i get it on my site as well yet it works on my programmers site which is the same program.

http://www.modlow-arvai.com

Agagooga said...

ISPs' transparent proxies (e.g. Singapore) do that too

jthomson said...

This started happening much more frequently for us.

We run two proxy servers each serving approximately 100 people. From crawling over the logs, it doesn't look like there's any illegitimate traffic triggering this.

It happens most frequently with people using Firefox's google search box. (I believe this makes multiple requests as you type into the box).

Anyway, it's becoming particularly annoying, and we'd appreciate any advice on how to avoid this.

I'm surprised that there's only 45 comments on this page. Does that mean that it's not actually that common?

Malditang Pinay said...

I use Google Chrome and no matter what I do (i.e. use the address bar or use the Google search homepage) I still get the sorry page. Why would Google think I'm a bot? Why would Google think Chrome is a spyware or malware?

Simon F said...

I have this annoyance. I run Linux and there is no automated software on my PC. If this this "We're sorry..." message persists I WILL switch search engine!

Agagooga said...

Try routing requests through CoralCDN:

http://www.coralcdn.org/

Syaifudin Zuhri said...

This was happening to me many time I hope I can find the answer here, and I got it .

Kathy said...

This blog was started in July of 2007. It is now January 31, 2009. Problem still occurring. Solution: Yahoo. I've had it.

AndreasNYC said...

Still blocking after all these years
My small company has been frequently hit by this Denial of (google) Service 'attack' and it's very troublesome. We checked our anti-virus & malware programs and scanned our computers and everything came up clean.

Having checked our computers, we've then requested review/help using the form google provides with: a) no results (still get denied service) b) no answer (they ask for an email address, but don't seem to use it).

How to get out of this mess? Do we all have to abandon google search? I hope not, but it's getting pretty annoying for me and my company/users.

Andreas

Matt Scarborough said...

I switched my default search engine to Cuil because of this "We're sorry" problem. From Google I don't ever receive a captcha. I am just dead in the water.

Recently I had narrowed my Google search results to 287 hits. All were relevant. The perfect query. Yet every time I clicked for page 3 of the results, I was thwarted by the "We're sorry..." dilema.

It think it unwise to make default use of a system that fails so clumsily in the face of a false positive.

Surely this blundering "We're sorry..." is Google's equivalent to the dreaded "Blue Screen of Death."

TtfnJohn said...

As has been noted there is no excuse at all for the massive number of false positives. Nor does the post bother to explain why or what Google may or may not do about it.

Not that it's a new problem or anything.

Not that a student in Summer of Code couldn't come up with a better protection scheme.

As Matt above has noted it appears to default to a false positive based on useage alone and not other well known indications of malware or bot and worm attacks.

What is so sad and frustrating is we all know Google can do better. Why it doesn't is the unanswered question.

ttfn

John

xxxxxxx said...

i trigger your we're sorry page once in a while.

I no longer look for malware on my computer because i have top of the line AV programs and have never found anything.

My big problem i have with the CAPTCHA is i fill out the box ansd then find no way to operate it.
there is no place on the page to click to get it to work.
and no i will not delet cookies
i get very upset having to log into all the site i am a member of because i had to delete cookies for your we're sorry page then find out that did not work ether.

another strange thing i have found is my computer is duel boot and all when i reboot into linix and try there i do not have a problem.

Jayme Snyder said...

I think we have made it trigger bassed on shear volume...

We have a customer with an international corporate WAN that leaves the internet from a single WCCP cache proxy farm. This centralizes content filtering but gives over 5000 users the same public address. They all get this error page :|

Donna Maher said...

I had never experienced this awful 'we're sorry' page phenomenon until today...

It's VERY frustrating, I've got all kinds of AV and spyware blockers and net security programs installed AND a firewall - so I doubt that my computer is infected and have no idea who else might be on the network since I'm using Comcast cable service.

All I know is that I cannot access my adsense account and it's not making me very happy since I need to get in there to do my webmaster work.

I see that nobody has a solution and this has been going on for over two years...

Is it possible that my recent change to VoiP service with Vonage could be the 'culprit' making Google's servers get wary? Not being the amazing techie that I'd like to be, I have no idea but am welcome to any suggestions here...

I did notice that my IP address changed after installing the Vonage adapter if that makes any difference?

Thanks,
Donna

Blog User said...

I get blocked from both Google and Reddit a few minutes after me and two others behind a firewall turn on our computers. Any of these computers (or the router) might be infected, but I have no cooperation from anyone to diagnose any of the systems except for mine: I have scanned my system's outgoing network traffic, and have found no evidence that my computer is the one generating the traffic.

Furthermore, Google provides no CAPTCHA whatsoever on the We're Sorry page-- there is simply no way to get past it. (Reddit pretends that it's down, giving me a "service temporarily unavailable" message from my IP address, but not from my shell account.

a said...

I think this "feature" is very buggy.

Some minutes ago I get the "sorry" message, but refused to fill the captcha (because it not helped at all in the past, and I have no worms etc.)

I've pressed the back button and searched for the sorry message with success (!) and landed here.

That means that some keyword trigger the message for me, and found that the guilty is at the moment is "asp".

So when I type ANY search term which contains "asp" like:
"asp.net bla bla" I get the sorry page, ALL other search works.

How is this possible?

David said...

I have started getting this today (Jun '09). Forgive me for sounding thick, but what is the answer to this??

Is this error message a legitimate Google Error page and is therefore safe to enter the code requested??

The google 'explanation' at the top of this blog is just techhy talk and doesn't give me an answer??

Or is it a virus on my pc??

Or a virus on Google??

And are Google trying to resolve it??

Lots of techhy talk but no simple answers given??

Can anyone explain for me in simple plain English??

wordlife said...

i dont believe google anymore.. i dont think they are showing the captcha because of some worm or something. they are just trying to keep the users in google even though their servers are down. or they are not able to get the related content at tat time. they are just confusing the users. if its abt worm then how come i got the captcha from my home network, office network and internet parlor network on aug 1st. and also google is changing its alogrithm when ever they want and not even informing in their blogs... as everybody know any of the sites are getting traffic from google they are just trying to get that traffic to the google sites... if my query contains vulnerable contents then how i will get the results after some time? google just accept that your API's are not properly tested...

Bill said...

I can't even use Google anymore. I Google a LOT and the past few days every single search I put in gives me that message I don't have time for that on every single search I do. I have now switched to another search engine. I can continue my searches as I used to in Google. Google was the best, but it's unusable now.

weweje said...

I'm here too to testimony that the sorry.google.com system is Buggy ( or far too sensitive )
I'm in a controled network ( traffic analysis/spyware activity detection etc...) and having an error on EACH cache request I make for 3 month or so, for now.
So the assertion " your access will be re-enabled soon " is fake.

Dear Google:
You are shooting in your own foot.
I swich to ASK.COM

weweje said...

You just dropped my precedent comment.
I ( and others on InterTube http://www.google.fr/search?hl=fr&um=1&q=%22%20we%27re%20sorry%22%20google&ie=UTF-8&sa=N&tab=iw )
think there is a huge issue with the system.

I'm working on a secured LAN, whose exit point is referenced @ spamhaus. Reason? we're hosting personal webspaces for free, so some phishers naturally lives there + we didn't replied to spamhaus, who is acting as if it had some sort of "legal" legitimacy or infailability.
The unique logical reason for getting those repetitive " we're sorry " messages upon our network is because of the Spamhaus listing. But eh, wait: we're just the 2nd ISP in France, so this does impact our customers too. You are denying regular search to a LOT of people, because you just trust Spamhaus for good.

Mikael said...

I'm a single user behind a linux/squid proxy server. I'm seeing the captcha once every day from each computer I log in from. If I don't go through the proxy, this problem doesn't happen. Is there some way to reconfigure my proxy to avoid this?

Mikael said...

Ok, please ignore my previous post. The proxy I was using really was being hijacked by bots. So, in my case; everything worked like it should :)

weweje said...

Hello there?
Could a google sec-tech reply to our concerns; I mean, nobody is infailable, so if the system can be improved, let us know what is actually triggering the "we're sorry", or if you @ google indentified some problems with your detection system.
Thanks

ladyhawke999 said...

Hello,

it's been more than 2 weeks I'm not able to access my orkut account through my mobile phone because of CAPTCHA messages. I don't what's going on, I've always used the same phone, browser and provider to access it. I Use HTC Snap with windows mobile 6.5 and as browser Internet explorer. My telephony provider is 3 (from Italy).
I really hope someone here can help me.
Thanks in advance.

braden said...

Hello I am a motorola droid user on android 2.2 os and my theory is I searched too fast and I am now getting this message whenever I access google, I'm using this to do a school report since my computer broke and now I can do anything. I have no maleware or anything I've ran my virus software multiple times and even remove my sd memory. And it tells me there should be a captcha in the help page but on the error page there is no captcha please help.

Jeff said...

Been seeing this on iPhones lately when using mobile safari. So does AT&T have a problem?

Drake said...

Funny. I agree with some posters above whom mentioned ReCAPTCHA. After all, doesn't google finance reCaptcha? I thought that if you pay for it, you should use it... at least, that's what I learned in my five and a half years of business school...

Randy Saunders said...

This has become an ongoing problem for AT&T iPhone users. Since there are lots of iPhones, and a finite IP space with which to serve them, this can't be fixed. Google wants iPhone users to switch to yahoo, to reduce load on Google's servers.