I just added UserStream into my application and its overloading my server

Apr 25, 2013 at 4:42 AM
I am running the UserStream 24/7 on my iis web server and suddenly after close the browser and open it back up it takes longer and longer to load the site.

my suspect I am not closing the userstream the correct way.

I am also putting the userstream on a seperate thread using c# and .net

is there any way to kill the userstream after I close the website

here is the code i am using that the worker thread calls to start the stream

public void GetUserStream()

       TwitterContext  twitterCtxStream = new TwitterContext(this.TwitterAuthorization);

          DataContractJsonSerializer serializer3 = new DataContractJsonSerializer(typeof(_EventObject));


              (from strm in twitterCtxStream.UserStream
               where strm.Type == UserStreamType.User
               select strm).StreamingCallback(strm =>

                        // Also the Status is not an option?????
                      //if (strm.Status == TwitterErrorStatus.Success

                      //    return;

                      if (strm.Content != null && strm.Content.IndexOf("\"event\":") > -1)
                          _EventObject eventObj = serializer3.ReadObject(new MemoryStream(Encoding.UTF8.GetBytes(strm.Content))) as _EventObject;

                          if (eventObj.Event == "follow")
                            // do something


catch (ThreadAbortException)
        catch (Exception e)
  // CurrentSession.SendServerErrorMessage("UserStream: " + e.ToString());


            _isRunning = false;
            Setup(true);  // call the userstream again after 3 seconds

Apr 25, 2013 at 5:05 PM
Edited Apr 25, 2013 at 5:10 PM

You might be running into problems because of your design. Twitter streams are mechanisms that capture live traffic. This differs from a single query that reads tweets and other entities from the past. Therefore, your architecture has to be different. You might want to review Twitter's documentation on the Streaming API to understand the approach you should take.

With this in mind, I'm wondering if your choice of technology for using streams is the underlying problem. In the case of Web applications, the lifetime of the server objects in your code last for a single request. Then they go away. This means that if you're starting a stream during a request, as soon as the request completes, the objects used during that request become eligible for garbage collection. Therefore, your stream will stop working at an unpredictable time after the request completes. On the next request, you will open another stream, which will also stop working as soon as the request completes. After this happens enough times, Twitter might block your app/ip address for a given amount of time because it looks like you are continuously opening and closing streams. This will result in (4xx) error messages, which are translated into WebException and exposed through the Exception property of your callback. You want to check the status in your callback to know if Twitter sent you an error. If you generate too many exceptions, your app performance will degrade significantly.

When using streams, most people host the code in a long running process, such as a Windows Service or Azure Worker Role. This keeps the stream open and alive. It would be good to review Twitter's guidance also.

Update: For closing the stream, capture the param passed to the callback (strm) and call Close().

May 9, 2013 at 1:18 AM
Hi Joe!

Thank you now for your feedback. I want to know if I am going in the right direction.

Here is my scenario. I have people signing in on my site and they each are using the userstream and one of the features I offer is everytime someone follows them the application will updatestatus thanking the new follower. now I know this is a server nightmare.

Now I want to follow your advice and create a windows service or azure worker role.

and now that you know the scenario what would you suggest?

I know you love these crazy scenarios Joe!!

Code On!

www.Twambit.com powered by LinqToTwitter
May 9, 2013 at 1:22 AM
if I create a windows service
I imagine i have to create a new task or thread when a user requests Twambit.com

my question is can I pass the oauth and twittercontext parameters from my webapplication .aspx page to the windows service for each user?

or is this more like of an azure solution? forgive me for my lack of knowledge in either windows service and Azure web workers.

thank you Joe
May 9, 2013 at 1:33 AM

If it was just a simple application that performed a predictable task all the time, I would do a Windows Service. However, the traffic from Twitter can be unpredictable, users can change, and hopefully your business will continue to grow. A Windows Service won't scale that well - at least without more engineering. So, I would host in an Azure Worker Role. The benefits are the performance and scalability. You can add more worker roles as needed. You also have more growth opportunities through service bus and caching. Service bus messaging is the way to go, instead of Queues because the rates are cheaper, metrics are more understandable, and you'll have more flexibility on what you want to do with tweets. There's lots of help with Azure too and you can download the training kit and SDK for free and costs are reasonable because you only pay for what you use. I use LINQ to Twitter on Azure and it Rocks!