Search tweets by country

Jul 1, 2012 at 12:57 PM
Edited Jul 1, 2012 at 1:28 PM

Hi there,

Thanks for the great LINQtoTwitter library!!

I have a few questions I was hoping someone can help me with?

1)
I want to search all tweets, but only search tweets from Australia.  How might I do this?

I am currently using the GeoCode field on search - but I get tweets from all over the world.  I also look at the GeoCode of tweets retrieved by the search, but only 1/50 have anything in this field at all!

My code is below (if that helps).  I get results back, but have no way I know of to specify that I only want results from Australia!
Please, is there any way I can use LinqToTwitter (or anything else) to find this out?


2)
I wonder if the problem might be that I have not turned GeoEnabled to true.  I was looking for this in the object model, and the code examples, but couldn't find anything that shows me how to do it.
Do I need to access the Twitter library through my Twitter account?  Or does it mean I already have it on if I sometimes get GeoCodes back? (only 1/50 times though!)


3)
Is there any other location mechanism other than GeoCode?  Can I search by country or city or anything?

 

4)
(Just a little question).  I like the idea of the search by "Attitude".  But once I have retrieved a search result (without searching by attitude) how can I tell what the attitude of the tweet was?  Can I access that information?

 

5)
(One more little question).  Can I make the search 'case sensitive'.  So searching for "BHP" will never return bhp?


THANKS FOR YOUR HELP :)

(If anyone can offer any help that is).

Kind regards,
Donald

 

my code:


        DateTime querysince = DateTime.Today.AddDays(-1).Date;
        string geocode_melb = "37.7833,144.9667,60km";
        int num_pages = 1; int num_per_page = 50;

        (from twtsearch in twitterCtx.Search
            where twtsearch.Type == SearchType.Search &&
               twtsearch.Query == "BHP" &&                             
               twtsearch.Since == querysince &&
               twtsearch.PageSize == num_per_page &&
               twtsearch.ShowUser == true &&
               twtsearch.GeoCode == geocode_melb &&
               twtsearch.Locale == "en" &&
               twtsearch.Page == j

         select twtsearch)
            .AsyncCallback(searchResponse =>
            {
                Search srch = searchResponse.Single();

                k = 0;
                foreach (var entry in srch.Results)
                {
                   

                    //act on the search result here
                    fileContents += "\t\t" + entry.Text + "\n";
                    if (entry.Geo.Coordinates.Count == 0)
                        fileContents += "\t\t\n";
                    else
                        fileContents += "\t\t" + entry.Geo.Coordinates[0].Longitude + ":" + entry.Geo.Coordinates[0].Latitude + "\n";


                }
            }).SingleOrDefault();
Coordinator
Jul 1, 2012 at 11:20 PM

Hi Donald,

It looks like you're doing everything correctly - I'll answer each question in order:

1) Your GeoCode parameter is in the correct format. However it looks like it's in the middle of the Pacific Ocean off the coast of Japan.  You might want to open a discussion with Twitter to see if they support your use case.

2) I'm not sure what you're referring to by GeoEnabled.  There isn't a query parameter that can be set for that.  There is a setting on a user's account for including their location in tweets, but that can only be set by that user and is only applicable to their own tweets.

3) In addition to GeoCode, there appears to be a way to search by Place ID: https://dev.twitter.com/docs/places/finding-tweets-about-places.  It seems like you can use LINQ to Twitter's Geo queries to find place information.  Once you have place info then you can specify your query as:

... twtSearch.Query == "place:criteria"  i.e. "place:247f43d441defc03" or "place:opentable:2"

where a colon separates place and whatever you use for criteria, such as a place ID.

4) I'm not aware of anything in the results that indicates attitude or any other way to specify it other than the query parameter.

5) I wasn't able to get case-sensitive results.  I tried "\"BHP\"", which encoded properly into the query, but the results were still case-insensitive.

@JoeMayo

Jul 2, 2012 at 12:32 AM

G'day Joe,

Thanks really so much for your help!  And quick response too, very happy.

Just a few quick follow-ons.

1)

Fixed it!  Turns out my address was defined as "south" rather than "north", which twitter represents as a negative.  I get my tweets searched from Melbourne (Australia) now! :)

2)

Fair enough.  Though, I had hoped to see more "GeoCodes" on people's tweets so I can tell where the tweet is from.  It seems really strange (is it true?) that I can search for tweets by location (specifying the geo string) - but then can't actually see the location of the tweet as is returned! 

I mean, for instance, I can find out which Australian city each tweet is from by searching Melbourne, then Sydney, then Adelaide, etc - but I can't tell by searching all of Australia, then using the GeoCode to find location?

Is there any way I can see the GeoCode for all (or at least 9/10) tweets?  Currently maybe only 1/50 I can see the GeoCode.  Searching 10 times to find approximate location inside Australia is a bit of a drag!

3)

THANKS for the tip!!  Awesome.  But what is an information source I can use to find "place IDs"?   Like the string to represent "Australia" for instance?   Also, what is this "opentable" stuff?  Any idea where I can look to find out more? 

4)

No worries!  Thanks for letting me know.

5)

That's a pain, but ok.  I'll try and do some googling on it.

 

Thanks again Joe, very very helpful.

Donald

Coordinator
Jul 2, 2012 at 2:10 AM

2) The Geo property in the SearchResults response only contains coordinates if the user turned on location in their personal profile.  I'm not sure how Twitter is determining the location of the tweets that don't have coordinates.

3) Use LINQ to Twitter's Geo queries:

http://linqtotwitter.codeplex.com/wikipage?title=Getting%20Geographical%20Information&referringTitle=Making%20API%20Calls

i.e. you could take the lat/long you have and do a GeoType.Reverse query.  The API even allows you to define your own place, which you can then use in your queries.

The examples I gave were from Twitter's Finding Tweets About Places page, at the link I provided.  I provided it so you could see the syntax of the query without having to figure out that the Url encoded %3A is a colon.

I think the Places approach that Twitter uses is modeled after Yahoo's Places and WEOID.  I'd scan the Twitter site for what info they have on places and also look at how Yahoo defines places.

Joe

Jul 3, 2012 at 12:40 AM

That's great Joe, thanks a lot!   Very helpful.

 

Playing around now finding an optimal way to do a lot of searches on Twitter- bringing back the data for a data mining project, so need to make a lot of searches quickly.

Unsure about how best to do this, am using the BackgroundWorker stuff to do multi-threading, and using the asynch seach on LINQtoTwitter.  Having some success, but will push forward on trial and error.

(If you have any advice into areas to look into that would be most useful!!)

 

Thanks again for your help,

Donald