This project has moved and is read-only. For the latest updates, please go here.

Problems with Streaming API

Jul 9, 2013 at 12:28 PM

I am experiencing a problem with the Streaming API. I am calling the sample method from StreamingDemo a bunch of times, but never is the streamingCallBack method called. It just won 't execute the lambda. Problems with authorization maybe? I don't get an error, it just won 't go into the expression.

private static void SamplesDemo(TwitterContext twitterCtx)
            Console.WriteLine("\nStreamed Content: \n");
            int count = 0;

            (from strm in twitterCtx.Streaming
             where strm.Type == StreamingType.Sample
             select strm)
            .StreamingCallback(strm =>
                if (strm.Status == TwitterErrorStatus.RequestProcessingException)

                Console.WriteLine(strm.Content + "\n");

                if (count++ >= 10)
Jul 15, 2013 at 2:07 PM
Edited Jul 15, 2013 at 4:43 PM
Perhaps I should give some additional information:

I am getting the unauthorized error when analyzing with fiddler: < HTTP/1.1 401 Unauthorized
< Content-Type: text/html
< WWW-Authenticate: Basic realm="Firehose"
< Transfer-Encoding: Chunked

It happens with every kind of authorization I use. Do I need some extra authorization/authentication first with a username/password?

Below I added a link to a fiddler screenshot of a typical sequence of packages.

Anyone please help! I really need this to work

Jul 15, 2013 at 3:39 PM

First, your app is now insecure and you need to take down that image and update your Twitter app keys because now everyone who reads this can use your keys, which is insecure. The best way to handle these things is to post the text and replace your secrets with something other than the secret. e.g. Bearer <bearer token goes here>.

The problem here is that you're using ApplicationOnlyAuthorizer, which doesn't work with streams. You should try SingleUserAuthorizer. I just (yesterday) updated the sample for SingleUserAuthorizer, which you can find in the LinqToTwitterDemos project in the downloadable source code.

If you're getting 401's, you can read the FAQ, which contains several tips that might help.

Jul 15, 2013 at 4:32 PM
Hi Joe,

Thanks a lot! The FAQ was very helpful, i did try the single user, but I didn 't use the proper singleuserinmemory credentials. Now I am receiving status 200 messages from the sample api, but stillI can 't seem to print the content in the callback function. Thtere is some magic in the lambda, I seem to be missing, it does 't seem to wait till the message arrives and so seems out of sync. Perhaps you have a suggestion?

Thanks, really appreciate it

Gr. Joost
Jul 15, 2013 at 4:56 PM
additional info: - The lambda code just does 't seem to be executed, never is a breakpoint hit. It just calls the streaming api and ignores the StreamingCallback function. Maybe a version-thing?
Jul 18, 2013 at 1:08 PM
Edited Jul 18, 2013 at 2:13 PM
Have you got a Console.ReadKey in your app to stop it exiting?

The issue I have is it breakpoints on Console.WriteLine(strm.Content + "\n"); but doesnt write to the Console and never hits the count check.

UPDATE: I git cloned the source and ran the demo and selected "2" for single auth and put my details in app.config and it threw 401 exception. The details are correct as I have L2T working in another app.

UPDATE 2 : Got it working by using twitterCtx.Log = Console.Out; I asume thats intercepting Console.WriteLine somehow
Jul 18, 2013 at 1:57 PM
I have my app working. I am doing data analysis, so I basically am doing an infinite loop to catch as many tweets as possible. I start off with letting the thread sleep for like a minute and it will call the streaming api and sleep again. In Fiddler you will see the messages coming in. The stream is being downloaded. Because my app does nothing for a while, it grows to 1.5 MB. Then it calls the streaming-code again and it 'eats' away the stream. The sleeping is important, because otherwise you will call the Twitter API too many times and you will exceed some rate limit. Below my code:
while (true)
                Console.WriteLine("First do nothing");
                Thread.Sleep(90 * 1000);
                    using (twitterCtx = new TwitterContext(getAuthorizer()))
                        int counter = 0;
                        string content = "";

                        query = (from strm in twitterCtx.Streaming
                                 where strm.Type == StreamingType.Filter &&
                                 strm.Locations == "-116.97,32.82,-73,41,-122.75,36.8,-81.376701355,48.5696983337"
                                 select strm).StreamingCallback(strm =>
                                     if (counter++ > 800 || strm.Status == TwitterErrorStatus.RequestProcessingException)

                                         string temp = content;                                         
                                         counter = 0;
                                                   //Here is where you save the content / temp. i used JSON.Net : JObject.Parse ... work very easy
                                         content = "";

//I noticed that each time i call strm.content it contains 1 tweet, but every third tweet is garbled for some reason. so I filter:
                                     if (!strm.Content.StartsWith("{\"created") || !strm.Content.EndsWith("\"}"))
                                         content += strm.Content;