Google Query from Java

Go To


I am using this code to do a query over list of words using google and extract number of search results from google. It worked fine, but since last night it keeps giving me this error after doing 200 queries (I guess google flagged me!): Exception in thread "main" Server returned HTTP response code: 503 for URL:

the "red" is just an example.

public class Google {
public static void main(String[] args) throws IOException {
        String query = "red";
        String urlName = "\""+query+"\"";
        URL url = new URL(urlName);
        URLConnection conn = url.openConnection();
                "Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv: Gecko/20070723 Iceweasel/ (Debian-");
        BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
        Pattern pattern = Pattern.compile("<div>About (.*?) results</div>"); //<div>About 1,620,000 results</div>
        String line;
        while ((line = in.readLine()) != null) {
            Matcher m = pattern.matcher(line);
            if (m.find()) {
                System.out.println(; // coresponds to results number: i.e.: 1,620,000


any solution, suggestion?

2012-04-04 20:54
by Marcus
I found and easy and yet not the best solution. just add Thread.sleep(15000); between each queries. this way it would delay 15 seconds between each queries and google will not take it as a robot - Marcus 2012-04-06 03:18


You've been flagged as a bot, probably due to the frequency of your query. Try running this from a different IP (before that one is flagged as a bot).

Regardless, you should probably use the Google search API. From the site


Free quota

Usage is free for all users, up to 100 queries per day.

Paid Usage

Any usage beyond the free usage quota will fail if you are not signed up for billing. Once you have enabled billing, you will continue to receive 100 free queries per day. However, you will be billed for all additional requests at the rate of $5 per 1000 queries, for up to 10,000 queries per day. If you need additional quota, please request additional quota from the console.

2012-04-04 21:03
by Steve H.
I did 42,000 queries and it worked fine. I need to cover another 110,000 queries! and With 100, I will need 1100 day! any way, I will need to fine another way - Marcus 2012-04-04 21:39


From the error page specifically:

"Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot."

Since you obviously are a robot hitting their page they've put in certain measures to circumvent the traffic you're costing them an do not wish for you to continue such a practice.

Having said that you'll need to verify your identity with Google at some point. The way this page recommends doing so it to present the author (yourself) with the image, manually bypass the captcha and then save the cookie for use by your program.

2012-04-04 21:02
by Grambot
I'd wager that actually goes against their ToS, especially since you are explicitly circumventing their abuse service - James 2012-04-04 21:10