A few search questions

Oct 24, 2011 at 9:40 AM

Hey there!

Firstly, thanks for LinqToTwitter, it has made my life alot of fun over the last few weeks, I am working on a cool little project! I have question however if you dont mind...


1. When searching I can specify which page to return... however as far as I can see, I am never told the total number of pages available, how would one go about checking to make sure I have collected all results? is this a case of simply checking the number of entries and stopping when one is turned with less than the number of maximum results?


Many Thanks


Oct 24, 2011 at 2:09 PM

Hi Ben,

There aren't any API's that indicate the number of pages in a given request. Twitter started out with the concept of using pages for it's API, but later migrated to using cursors in several places, which is based on a page-by-page request mechanism.  I recall the reasoning for this approach derived from the opportunity for better performance and scalability.  In some places, such as requesting tweets, knowing how many pages are available changes by the second because there are always new tweets arriving.

I think your approach for pulling results is fine and I have a couple more ideas for you.  You might want to request until you receive no results back because each request, depending on API, might not give you back the exact number of results.  Another thing you might do is use SinceID (or similar parameters), where available, to keep track of the last place you were making requests so you can continue where you left off and not keep requesting the same results.  If you're requesting a lot of pages, you might encounter certain limits per hour that you can check by looking at RequestHeaders or related properties in TwitterContext.  Some of these items might not apply to you, but I mention them just in case it might help.