Searching all tweets with a specific url

Aug 20, 2012 at 2:07 PM

I don't know if this question has been asked before, I searched but could not find.

What I need to do is search for last 100 tweets which have my website url  in them.

All my products have links like this: http://urlname.com/Product/ProductName1

So I need to find tweets which has 'http://urlname.com/Product/' in them.

 

Thanks

Arnab

Coordinator
Aug 20, 2012 at 4:29 PM

Hi Arnab,

You can use the Query and PageSize parameters in a Search query, like this:

            var srch =
                (from search in twitterCtx.Search
                 where search.Type == SearchType.Search &&
                       search.Query == "http://urlname.com/Product/" &&
                       search.PageSize == 100
                 select search)
                .Single();

            Console.WriteLine("\nQuery: {0}\n", srch.QueryResult);
            srch.Results.ForEach(entry =>
                Console.WriteLine(
                    "ID: {0, -15}, Source: {1}\nContent: {2}\n",
                    entry.ID, entry.Source, entry.Text));

Here's the documentation for more information: http://linqtotwitter.codeplex.com/wikipage?title=Searching%20Twitter. BTW, I corrected the documentation after seeing your question - thanks.

Joe

Sep 17, 2012 at 2:56 PM
Edited Sep 17, 2012 at 2:57 PM
Tx Joe, but srch does not have any QueryResult or Results,
Can you plz chk again... or else I must be doing some very silly mistake...
Coordinator
Sep 17, 2012 at 4:14 PM

Hi,

Did you, by chance, download the latest source code?  If so, I can see where this could be confusing.  I'm currently upgrading LINQ to Twitter to Twitter API v1.1 and the Search API changed a lot.  In particular, it returns a response via the Statuses property, which is a List<Status>.  The newest bits haven't been released yet and I'm still working on them.  The current release, via the downloads tab, should be consistent with the documentation.

Joe

Sep 17, 2012 at 7:45 PM

ah, ok, tx..

Sep 17, 2012 at 8:21 PM

current release allows me to view results and queryresult but does not give right search results..

I looked for http://www.giftsocial.in/product     as well as giftsocial.in/product

It gave results count=0

but there are two tweets I have done which contain these urls..

https://twitter.com/c_arnab/status/238272609098297345

https://twitter.com/c_arnab/status/238272609098297345

 

Thanks

Arnab

Coordinator
Sep 17, 2012 at 9:23 PM

I think it's because Twitter isn't returning results.  This page doesn't return results for those queries either: https://twitter.com/#!/search-home. I was going to recommend that you try a couple of the different search query options to target the contents of the tweet specifically, but the Advanced search, at the URL above doesn't return anything there either.  It does sound strange why these tweets won't show in search, but maybe you can post to https://dev.twitter.com/ to see if they can explain why.

Joe

Sep 20, 2012 at 6:19 AM

I had asked as per your suggestion ( https://dev.twitter.com/discussions/11037 ) and was told to use the streaming api.

Can I use linq2twitter to do this..

Thanks

Arnab

Coordinator
Sep 20, 2012 at 7:38 AM

That's what I thought, but couldn't remember how to find that FAQ.

Yes, LINQ to Twitter supports Twitter streams.  It sounds like you might want to use a Filter stream.  There are LINQ to Twitter examples in the source code and documentation available.

You should also read Twitter's Stream API documentation to understand what features are available and how they fit your requirements.  While LINQ to Twitter encapsulates the plumbing of communicating with the stream, you'll still want to learn about Twitter's rules for using the stream.

Joe

Oct 7, 2012 at 8:20 PM
Edited Oct 7, 2012 at 8:21 PM

I'm trying to use streaming api in asp.net mvc demo

 

public ActionResult Index()
        {
            string result="";
            if (credentials.ConsumerKey == null || credentials.ConsumerSecret == null)
            {
                credentials.ConsumerKey = ConfigurationManager.AppSettings["twitterConsumerKey"];
                credentials.ConsumerSecret = ConfigurationManager.AppSettings["twitterConsumerSecret"];
            }

            auth = new MvcAuthorizer
            {
                Credentials = credentials
            };

            auth.CompleteAuthorization(Request.Url);

            if (!auth.IsAuthorized)
            {
                Uri specialUri = new Uri(Request.Url.ToString());
                return auth.BeginAuthorization(specialUri);
            }

            twitterCtx = new TwitterContext(auth);
            twitterCtx.StreamingUserName = "xxxxxxx";//I'm using my twitter username
            twitterCtx.StreamingPassword = "xxxxxxx";//my twitter password

            var friendTweets =
               (from strm in twitterCtx.Streaming
                where strm.Type == StreamingType.Filter &&
                    strm.Track == "twitter"
                select strm)
                .StreamingCallback(strm =>
            {
                result+=strm.Content;
            })
            .SingleOrDefault();
            return this.Content(result);
        }


It goes to twitter authentication page AND then comes to friendTweets function but does not return anything ('result' is empty)
Thanks
Arnab 

Coordinator
Oct 7, 2012 at 9:43 PM

Hi,

Using streams like this won't work.  The issue is the stateless nature of Web applications.  i.e. you do a request, the site processes and responds to that request, and then the site releases the request resources.  So, you might successfully start the stream, but all the resources used go away when the instance of the controller containing them are released. Even if there is some type of reference to the stream keeping the object alive, the next request builds a brand new controller object, unaware of what the previous request had.  There are many more reasons why this won't work, but that's just a quick explanation of what's happening.

For streams to work, you need a service running continuously.  The nature of the stream is that LINQ to Twitter receives Twitter content via the stream in real-time and gives that content to your application to work with.  This is unlike the REST API, where you make a query and receive a single result set.  Your Web site can then communicate with that service.

Joe

Oct 8, 2012 at 9:45 AM
Edited Oct 8, 2012 at 9:50 AM

ok, but all twitter dev accounts are tied with a specific url in my dev environment its http://127.0.0.1:30326/ , so in my mvc app I state the port no to be 30326 and it works but in case of service what do I do?

Also I tried the streamingdemo in linqtotwitterdemo (I tried the xauth-option3) but got an 401 unauthorized error.

Also, stacktrace seems to go to LinqToTwitterAg, is this project also necessary reference?

-------------------------------------------------------------------------------------------------------------------------------

System.Net.WebException was unhandled  Message=The remote server returned an error: (401) Unauthorized.  Source=LinqToTwitter  StackTrace:       at LinqToTwitter.OAuthTwitter.WebResponseGet(HttpWebRequest webRequest) in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterAg\OAuth\OAuthTwitter.cs:line 525       at LinqToTwitter.OAuthTwitter.WebRequest(HttpMethod method, String url, String authHeader, IDictionary`2 postData) in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterAg\OAuth\OAuthTwitter.cs:line 445       at LinqToTwitter.OAuthTwitter.OAuthWebRequest(HttpMethod method, Request request, IDictionary`2 postData, String callback) in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterAg\OAuth\OAuthTwitter.cs:line 368       at LinqToTwitter.OAuthTwitter.PostAccessToken(Request request, IDictionary`2 postData, String& screenName, String& userID) in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterAg\OAuth\OAuthTwitter.cs:line 141       at LinqToTwitter.XAuthAuthorizer.Authorize() in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterAg\OAuth\XAuthAuthorizer.cs:line 26       at LinqToTwitterDemo.Program.DoXAuth() in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterDemo\Program.cs:line 163       at LinqToTwitterDemo.Program.ChooseAuthenticationStrategy() in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterDemo\Program.cs:line 97       at LinqToTwitterDemo.Program.Main() in D:\projects\000ecomNfacebook\linqtotwitter-61e6369ac576\LinqToTwitterDemo\Program.cs:line 13       at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)       at System.Threading.ThreadHelper.ThreadStart()  InnerException: 

-------------------------------------------------------------------------------------------------------------------------------

Also the app.config has three options after consumerkey and secret

<add key="twitterOAuthToken" value="" /> 

  <add key="twitterAccessToken" value="" />

    <add key="ClientSettingsProvider.ServiceUri" value="http://127.0.0.1:30326/" />

I added the dev url in serviceuri, is that correct?, what do I add in the other two..

In my dev.twitter.com application page , I found two values at bottom of page- access token & access token secret, are these the values..

It would be great, if you could provide an windows service example with streaming api ..

Thanks

Arnab

Coordinator
Oct 8, 2012 at 12:41 PM

Here's a nice project that uses LINQ to Twitter streaming:

https://github.com/postworthy/postworthy/tree/master/Postworthy.Tasks.Streaming

You can create a Windows Service to host the code.  Use SingleUserAuthorizer and add all 4 credentials, from your registered Twitter Application, to the config file.

Joe

Jan 16, 2014 at 6:27 PM
Joe, I wanted to say thanks for mentioning my Postworthy tool. I have put in a lot of effort and have made good use of the LinqToTwitter project also. LinqToTwitter is great!

-Landon