How to return tweets created by the new Retweet feature (Beta)

Dec 17, 2009 at 11:07 AM

I need to return all Tweets from my ScreenName, including tweets created using Twitter's new new Retweet feature (Beta).

At present when I select all tweets by ScreenName, these special Retweets are not included in the results returned. I can find no way to get at these Retweets. Can anybody help?

Below is an example of the method I've written to get tweets by ScreenName (e.g. "timacheson" is the ScreenName of my personal Twitter account).

On a separate note, today I noticed that this method is consistently returning 4 fewer tweets than it should do. Perhaps this is another bug in Twitter. Can anybody explain this or help me to understand why?


        public List<Status> GetTweetsByScreenName(string screenName, int tweetCount)
        {
            var statusTweets =
                from tweet in new TwitterContext().Status
                where tweet.Type == StatusType.User
                      && tweet.ScreenName == screenName
                      && tweet.Count == tweetCount
                select tweet;

            return statusTweets.ToList();
        }

 

Coordinator
Dec 18, 2009 at 3:33 AM

Hi Tim,

The retweet API includes the following queries:

RetweetedByMe - Retweets by the logged in user

            var myRetweets =
                from retweet in twitterCtx.Status
                where retweet.Type == StatusType.RetweetedByMe
                select retweet;

 RetweetedToMe - Retweets from logged in user's followers

            var myRetweets =
                from retweet in twitterCtx.Status
                where retweet.Type == StatusType.RetweetedToMe
                select retweet;

RetweetsOfMe - Retweets about the logged in user

            var myRetweets =
                from retweet in twitterCtx.Status
                where retweet.Type == StatusType.RetweetsOfMe
                select retweet;

Retweets - All retweets of a specified tweet

            var friendTweets =
                from tweet in twitterCtx.Status
                where tweet.Type == StatusType.Retweets &&
                      tweet.ID == "5769361742"
                select tweet;

Regarding your last question about missing 4 tweets; For backwards compatibility, Twitter doesn't include Retweets in the return results.  If you check what tweets were retweets, you'll probably find that there are 4 within the count.

Joe

Dec 18, 2009 at 10:00 AM

Is it possible to get retweets for a specified user (e.g. by ScreenName) without logging in as them?

Are there any code examples for merging Tweets and Retweets? Presumably this is the only way to get a list that matches the actual web feed for a particular user.

Thanks for the code examples, Joe, this is very useful, I'll try those and see what I can get.

Coordinator
Dec 19, 2009 at 4:11 AM

Getting retweets into the user timeline seems to be a popular request for Twitter.  I'll support it as soon as available.

You can do a union of tweets and retweets like this:

        /// <summary>
        /// Shows how to get tweets and retweets by the logged-in user through a union
        /// </summary>
        /// <param name="twitterCtx">TwitterContext</param>
        private static void GetAllTweetsAndRetweetsDemo(TwitterContext twitterCtx)
        {
            var myTweets =
                (from tweet in twitterCtx.Status
                 where tweet.Type == StatusType.User
                      && tweet.ScreenName == "JoeMayo"
                 select tweet)
                 .ToList();

            var myRetweets =
                (from retweet in twitterCtx.Status
                 where retweet.Type == StatusType.RetweetedByMe
                 select retweet)
                 .ToList();

            var allTweets = myTweets.Union(myRetweets);

            allTweets.ToList().ForEach(
                tweet => 
                {
                    if (tweet.Retweet == null)
                    {
                        Console.WriteLine(
                            "Name: {0}, Tweet: {1}\n",
                            tweet.User.Name, tweet.Text);
                    }
                    else
	                {
                        Console.WriteLine(
                            "Name: {0}, ReTweet: {1}\n",
                            tweet.Retweet.RetweetingUser.Name, tweet.Retweet.Text);
	                }
                });
        }

Notice how I materialized the queries for myTweets and myRetweets with a ToList() before doing the union.  This pulls the results into a List<Status> where you can use LINQ to Objects.  Support for the entire suite of LINQ Standard Query Operators is on the radar.  Until then, you can accomplish what you need with this same technique of materializing results and finishing off with LINQ to Objects.

Joe

 

Dec 19, 2009 at 8:20 PM
Edited Dec 19, 2009 at 8:21 PM

Is there no way to get the retweets for a specified user? I think the StatusType.RetweetedByMe filter will only work if I am logged-in as the user who retweeted. I want to get the complete timeline for other users too, both tweets and retweets, just as I would see on twitter.com.

I like the Union solution, I'll use that, I can always sort the list if I need to.

Thanks so much for your help!

Dec 21, 2009 at 1:19 PM

I've tried various approaches, but cannot produce a query that returns retweets for a particular user. Instead I get an exception, typically the standard "Error while querying Twitter" error.

Here is an example of code I've tried to produce the method I need:

        public List<Status> GetRetweetsByUser(string screenName, int retweetCount)
        {
            try
            {
                var statusRetweets =
                    (from retweet in new TwitterContext().Status
                     where retweet.Type == StatusType.Retweets &&
                           retweet.Retweet.RetweetingUser.ScreenName == screenName
                     select retweet)
                     .Take(retweetCount);

                return statusRetweets.ToList();
            }
            catch (Exception e)
            {
                return null;
            }
        }

 

Coordinator
Dec 22, 2009 at 2:37 AM

Hi Tim,

Remember that you must be logged in as the user you're getting Retweets for.  The Twitter API only supports querying retweets of the logged in user. 

I haven't added the Take operator yet, but you can use the Count property, which Twitter understands.  Each of the API's tends to have a different approach to paging. Some use page/count, others  and Twitter has more recently migrated toward using Cursors.  My guess is that Twitter will evolve more API's toward using cursors.  Here's an example that would get the last 5 retweets of the logged-in user:

            var myRetweets =
                from retweet in twitterCtx.Status
                where retweet.Type == StatusType.RetweetedByMe
                   && retweet.Count == 5
                select retweet;

            myRetweets.ToList().ForEach(
                retweet => Console.WriteLine(
                    "Name: {0}, Tweet: {1}\n",
                    retweet.Retweet.RetweetingUser.Name, retweet.Retweet.Text));

This returns tweets from the first page and you can filter on the page property (retweet.Page == pageNo) to page through the results.

Joe

Dec 22, 2009 at 7:07 AM
Edited Dec 22, 2009 at 7:22 AM

Ok, thanks Joe, I'll do that. This is a great product!

I hadn't realised that this was a limitation of the Twitter API. IMHO it's a shame you have to log-in to get re-tweets! Retweets are arguably part of a user's timeline. Logging in to get my own complete timeline seems an unnecessary extra step, and of course this means feeds retrieved for other users will always be potentially incomplete (unless we happen have their password). :)

This is the problem:

statuses/user_timeline
Returns the 20 most recent statuses posted from the authenticating user. It's also possible to request another user's timeline via the id parameter. This is the equivalent of the Web / page for your own user, or the profile page for a third party.
Note: For backwards compatibility reasons, retweets are stripped out of the user_timeline when calling in XML or JSON (they appear with 'RT' in RSS and Atom). If you'd like them included, you can merge them in from statuses retweeted_by_me.

Source: Twitter REST API Method: statuses/user_timeline

Dec 23, 2009 at 10:03 AM
Edited Dec 23, 2009 at 11:46 AM

I spoke to some Twitter API guys who asked me to submit this as an issue, which I have done.

Twitter API issue reported:
http://code.google.com/p/twitter-api/issues/detail?id=1312

Thread in Twitter API discussion group:
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/3b019839bf0e4eda

Other developers have complained about this in the Twitter API discussion group, and at least one other issue has been raised. The issue I saw was an Enhancement request, while one might argue that this is more of a Defect because expected functionality is not present.

Coordinator
Dec 23, 2009 at 12:13 PM

I agree.  I've visited the link and starred the issue.  I recall reading somewhere that Twitter looks at issues that have the most stars as being more important, so if anyone else encounters this problem, they should visit the link and star the issue too.

Joe

Dec 24, 2009 at 10:17 AM

It sounds like this issue will indeed be actioned as an Enhancement. This is good news:

http://code.google.com/p/twitter-api/issues/detail?id=1312

May 27, 2010 at 7:21 AM
Edited May 27, 2010 at 8:23 AM

Good news! Twitter fixed this issue today! There's a new parameter for including retweets in timelines.

This fulfils my request and fixes the issue I reported.

Can we have retweets in timelines it in Linq2Twitter now? :)

Coordinator
Jun 15, 2010 at 2:06 PM

Implemented in ChangeSet #67687.

Joe

Jun 15, 2010 at 2:19 PM

Awesome, thanks! :)