Streaming with hashtag support

Nov 3, 2011 at 3:21 PM

Hi,

What I wish to achieve is a stream for a particular hashtag, which is refreshed for example every 5 seconds.

I looked at the examples and the best one I could find for my particular scope was this code :-

 

            using (var twitterCtx = new TwitterContext())
            {
                twitterCtx.StreamingUserName = "myusername";
                twitterCtx.StreamingPassword = "mypassword";

                try
                {

                    var streaming = (from strm in twitterCtx.Streaming
                                     where strm.Type == StreamingType.Filter &&
                                         strm.Track == strHashTags
                                     select strm)
                      .StreamingCallback(strm =>
                      {
                          Console.WriteLine(strm.Content + "\n");

                          if (count++ >= 2)
                          {
                              strm.CloseStream();
                          }
                      })
                      .SingleOrDefault();

                }
                catch (Exception exc)
                {
                    string exception = exc.Message;
                }
            }

When I execute this code, I am getting a result in var streaming, however I do not know how to get the particular images and content of the tweets.

Before I was using this code :-

            var twitterCtx = new TwitterContext();
            List<Tweet> tweetContentList = new List<Tweet>();

            var queryResults =
                from search in twitterCtx.Search
                where search.Type == SearchType.Search &&
                      search.WordAnd == strHashTags &&
                      search.WithLinks == true
                select search;

            foreach (var search in queryResults)
            {

                foreach (var entry in search.Entries)
                {
                    Tweet tweetContent = new Tweet();
                    tweetContent.TweetImage = entry.Image;
                    tweetContent.TweetText = entry.Content;
                    tweetContent.Language = entry.Language;
                    tweetContent.Location = entry.Location;
                    tweetContent.Created = entry.Published;
                    tweetContent.Source = entry.Source;
                    tweetContent.TweetTitle = entry.Title;

                    tweetContentList.Add(tweetContent);
                }
            }

However this does not refresh, or am I missing something?

Thanks for your help and time

Coordinator
Nov 4, 2011 at 10:11 PM

Hi,

The content is coming back as JSON, so you'll need to parse it into objects.  I've been planning on adding this capability, but haven't gotten to it yet and it's probably a while before I do.  One of the more popular JSON parsers is JSON.NET, which is another project on CodePlex.com.

@JoeMayo

Nov 8, 2011 at 10:01 AM

Hi Joe

I am trying to run the first part of the code again

using (var twitterCtx = new TwitterContext())
            {
                twitterCtx.StreamingUserName = "myusername";
                twitterCtx.StreamingPassword = "mypassword";

                try
                {

                    var streaming = (from strm in twitterCtx.Streaming
                                     where strm.Type == StreamingType.Filter &&
                                         strm.Track == strHashTags
                                     select strm)
                      .StreamingCallback(strm =>
                      {
                          Console.WriteLine(strm.Content + "\n");

                          if (count++ >= 2)
                          {
                              strm.CloseStream();
                          }
                      })
                      .SingleOrDefault();

                }
                catch (Exception exc)
                {
                    string exception = exc.Message;
                }
            }

Should the count in the var streaming be bigger than 0?  Because my count is always 0, so I am thinking that actually its not returning anything.
Thanks

Coordinator
Nov 8, 2011 at 2:08 PM

Really, var streaming shouldn't be there because it will never contain values.  The content stream is being received via the lambda assigned to the StreamingCallback extension method.

Joe

Nov 8, 2011 at 2:22 PM

Hi Joe

strm.Content is always empty though

I replaced Console.WriteLine(strm.Content + "\n"); with string streamContent = strm.Content; but this is always empty!

Coordinator
Nov 8, 2011 at 2:49 PM

I usually test like this:

  1. Start the program and set a breakpoint in the StreamingCallback handler.
  2. Tweet a new message that contains the hashtag assigned to Track.
  3. See if you hit the breakpoint and verify that Content contains the tweet you just made.

Joe

Nov 8, 2011 at 3:07 PM

 

Hi Joe

Did not quite understand exactly what you meant.
Basically I have an application, and I want to pass a hashtag to the code I pasted, and get a stream which I can then put into an object and output into a ListView.  How can I tweet the new message that contains the hashtag?  I am getting the hashtag from the db for a particular search and passing it to strHashTags, however strm.Content is always blank.
If I put a QuickWatch on var stream, in TwitterExecuter -  Last Url, I am getting http://stream.twitter.com/1/statuses/filter.json?track=foxnews.  
Basically what I want to achieve, is that I want to display the tweets for a particular hashtag, and the application automatically refreshes whenever there is new content, if that is possible.
Thanks a lot for your time
Coordinator
Nov 8, 2011 at 7:08 PM

The stream is live, which means you have to be connected to see what is being sent in real-time.  So, start you app, go to your Twitter client and post a status containing the keyword that your app is looking for, and then you should see your tweet because your application is listening.

Joe

Nov 9, 2011 at 10:45 AM

Hi Joe

still no luck!  So I started my app, basically at the moment I have a basic aspx page with a textbox so that I can monitor the stream

So my code is as follows :-

        public static int Count;
        public string strContent;

        public void Stream()
        {

                var auth = new AnonymousAuthorizer();
                
                using (
                var twitterCtx = new TwitterContext(auth, "https://api.twitter.com/1/",
                                                        "https://search.twitter.com/"))
                {
                    twitterCtx.StreamingUserName = "";
                    twitterCtx.StreamingPassword = "";

                    if (twitterCtx.StreamingUserName == string.Empty ||
                        twitterCtx.StreamingPassword == string.Empty)
                    {
                        Console.WriteLine(
                            "\n*** This won't work until you set the StreamingUserName and StreamingPassword on TwitterContext to valid values.\n");
                        return;
                    }

                    Console.WriteLine("\nStreamed Content: \n");
                    int count = 0;


                    var stream = (from strm in twitterCtx.Streaming
                                  where strm.Type == StreamingType.Filter &&
                                        strm.Track == "manchester united"
                                  select strm)
                        .StreamingCallback(strm =>
                                               {
                                                   strContent = strm.Content + "\n";

                                                   if (count++ >= 2)
                                                   {
                                                       strm.CloseStream();
                                                   }
                                               })
                        .SingleOrDefault();
                }
            }
        }

And the aspx I have txtStreamContent.Text = strContent.

Posted a tweet about Manchester United, but did not get any reply.  Also there are updates on this search almost every minute, however I am not getting anything.

Coordinator
Nov 9, 2011 at 4:03 PM

The reason this isn't working is because streaming won't work with a Web application.  The Web is stateless, which means that every time you make a request, all of your objects are instantiated, used, and then thrown away when the request completes.  There's state managment to help with this, but the basic problem here is that the stream needs to be connected to receive data and that isn't possible because of what I just explained.  You have a couple options.  The first option is to use the ReST API:

http://linqtotwitter.codeplex.com/wikipage?title=Searching%20Twitter&referringTitle=Performing%20Searches%20and%20Finding%20Trends

The second option would be to create a Windows Service that reads and collects tweets and your application can communicate with this service for new content.

If you were building a desktop application (or Windows Service as just mentioned), this wouldn't be an issue because your stream would be open as long as the program was running, but the Web is a different story.

Joe

Nov 9, 2011 at 6:13 PM

Hi Joe

I was using the WebApp just as a test to single out this method.

So basically my scenario is this.  I have a WCFDataService, where I have my edmx, basically a database for holding my data.  Then I have a model, which has a reference to this WCFDataService and I am doing all the Linq2Twitter calls in this Model.  So I have a class in this Model called actions, and I have the following method:-

        public void Gettream(string SearchTerm)
        {

            List tweetContentList = new List();

            int count = 0;

            using (var twitterCtx = new TwitterContext())
            {
                twitterCtx.StreamingUserName = "myusername";
                twitterCtx.StreamingPassword = "mypassword";

                try
                {

                    var streaming = (from strm in twitterCtx.Streaming
                                     where strm.Type == StreamingType.Filter &&
                                         strm.Track == "Manchester United"
                                     select strm)
                      .StreamingCallback(strm =>
                      {
                          Console.WriteLine(strm.Content + "\n");

                          if (count++ >= 2)
                          {
                              strm.CloseStream();
                          }
                      })
                      .SingleOrDefault();

                }
                catch (Exception exc)
                {
                    string exception = exc.Message;
                }
            }

        }

Now I am calling this method from my xaml in the Load Method

 

            Actions myActions = new Actions();           

           myActions.GeStream("Manchester United");

After going through the method, and not returning anything, the application just closes.

Nov 9, 2011 at 6:14 PM

Actually the method is called 

public void GetStream(string SearchTerm)
Sorry for the typo
Coordinator
Nov 9, 2011 at 6:26 PM

The WCF Data Service is the same as the Web app, it's stateless.  This is much different than a Windows Service.  The essential concept is that the TwitterContext instance must be continuously running for streaming to work.  On Web applications such as ASP.NET Web Forms, ASP.NET MVC, and WCF or ASMX Web services this is not possible.  You'll have to use the ReST API instead.

Joe

Nov 9, 2011 at 6:32 PM

Hi Joe

Thanks again for your reply.

So its either a Windows Service or else a REST API.  I will investigate REST API then since I have never done anything similar.  So for example an app like TweetDeck and other Twitter clients use a REST API right?

Coordinator
Nov 9, 2011 at 7:02 PM

I'm not sure what these other clients use, but the underlying concept is that you must be able to have a continuously connected instance of the stream open to use the streaming API.

Joe

Nov 10, 2011 at 6:33 AM

Hi Joe

Thanks a lot!  I will read about the REST API then.