This project has moved. For the latest updates, please go here.

Pulling all tweets from a single user with a date range

Feb 4, 2015 at 6:47 PM
I'm having trouble figuring out how to write a query to grab all Tweets from a specific account, within a date range (e.g., 2014-12-01 to 2015-01-01, essentially.)

I was looking at this thread, but .Since doesn't seem to be an attribute I can filter by anymore; there's a SinceID but I'm not sure how this is supposed to be used.

First, I was trying to write a query over Status, using CreatedAt.Ticks as criteria for my filter, but for some reason it's not returning anything from 2014.

lstTweets = await (
    from    tweet in _context.Status
    where   tweet.Type == StatusType.User &&
            tweet.ScreenName == ownerScreenName &&
            tweet.CreatedAt.Ticks >= start.Ticks &&
            tweet.CreatedAt.Ticks <= end.Ticks
    select tweet
Secondly, I was trying
lstTweets = await (
                from    tweet in _context.Search
                where   tweet.Type == SearchType.Search &&
                        tweet.Query == "@" + ownerScreenName &&
                        tweet.Until == end &&
                        tweet.SinceID == (ulong)start.Ticks
                select tweet
but still nothing is returning. I find it odd that .Until and .SinceID are of different data-types. Are these the parameters to use for "start" (Since) and "end" (Until) date?
Feb 5, 2015 at 1:23 AM
Edited Feb 5, 2015 at 7:20 PM

LINQ to Twitter changed since the discussion you're linking to. I was seeing unpredictable results from the Twitter API, so I removed properties for the advanced options. Instead, you can write Twitter's advanced options via text in the Query parameter. See the Twitter Search API for those options.

You might also need to page through the results because they could be more than what Twitter will return in a single query. I wrote a blog post on Working with Timelines with LINQ to Twitter. That article explains the role of SinceID and MaxID, but you should also read Twitter's Working with Timelines. The way to accomplish this makes sense, but is not immediately obvious, so you should also read Twitter's explanation of this technique too.

Feb 5, 2015 at 2:19 PM
Edited Feb 5, 2015 at 2:57 PM
OK, thanks a ton, Joe!

edit also your link to "Working with Timelines with LINQ to Twitter" is wrong, so here's the URL for any other people who happen across this thread for help:
Feb 5, 2015 at 3:57 PM
Edited Feb 5, 2015 at 3:58 PM
I don't know if this is related to the Twitter API or Linq2Twitter at all, but for some reason when I try to get Tweets, it's still not returning anything from 2014.

"from:xyz since:2014-12-01 until:2015-01-01" works if I'm searching FROM Twitter, but if I try that in the query field in my C# code I get nothing. If I remove the until field, it will return results from January and February this year. Are you familiar with any problems similar to this?

Should the query be returning something, or would I need to iterate through results (even though there are 0 results), similar to in the documentation about working with timelines?

Feb 5, 2015 at 4:48 PM
Hmm, I'm seeing various threads elseware that state that Twitter will only return search results within the past few weeks.
I'm going to keep searching for other ways to pull information about tweets for a user account within a date range - Maybe there's something that can be done. It would be really nice to be able to compile and build a report on Tweet statistics (retweets, favorites) over the span of a month or more...
Feb 5, 2015 at 5:24 PM
With querying on Statuses, there isn't a time limit, but it will return up to 3,200 results (GET statuses/user_timeline docs).

It looks like something like this is working out:

The only downside is that it isn't grabbing all of the Tweets from January, so if there really isn't a way around this, I'll just have to tell the client about the constraint.

Thanks for pointing me in the right direction!
Feb 5, 2015 at 7:26 PM
It looks like you're finding the correct information. There are many constraints on the Twitter API, which makes the task more complex. Another option, not free, is Gnip. Twitter acquired them a while back and they provide access to the entire history of tweets.

Thanks for the heads up on the incorrect link - fixed now.