How do I get 10k+ tweets streams in few minutes

Aug 7, 2014 at 12:50 AM
I am trying to get 10k+ streams using FilteredStream say for 2 tracks (I am not sure its right way or not), it works however process is very slow. What is right way to get 10k+ streams in few minutes?
Coordinator
Aug 7, 2014 at 9:54 AM
Hi there,

Well it all depends on what you want to do. If you want to filter tweets to get only one that are matching a specific criteria, you must use the FilteredStream.
With a FilteredStream I cannot guarantee how much tweet you will receive in few minutes. Using a FilteredStream you will need to add very generic terms that can be found in many tweets like "hello", "us", "france"...

If you do not care about filtering tweets, you can use the SampleStream. The SampleStream takes 1% of all tweets published on Twitter randomly and send them to you.
If you want to use the second one have a look in the first example of the documentation here.

Regards,
Linvi
Aug 7, 2014 at 5:44 PM
Thanks for quick reply.
I read your OneNote document on OneDrive. this OneNote has very good explanation of Streaminvi-> BaseTrackedStream. In performance section of this page it says 428,571.43 tweets per minute for 3 tracks each containing 3 words wow. I want to get tweets for 2 tracks each containing 1 word with same performance.
We can say I want to compare/analyze sentiments for 2 tracks for example microsoft and google or KFC and McDonalds. Please advice.

Thanks
Coordinator
Aug 8, 2014 at 12:41 AM
Hi,

When I did these tests, I had to take very popular keywords. Usually look into the news to get these kind of keywords.
Now if you want to analyze 2 tracks you can use the following.
var stream = Stream.CreateFilteredStream();
stream.AddTrack("MacDonald");
stream.AddTrack("KFC");
stream.MatchingTweetReceived += (sender, args) =>
{
    if (args.MatchingTracks.Contains("MacDonald"))
    {
        // Code logic for mac do
    }

    if (args.MatchingTracks.Contains("KFC"))
    {
        // Code logic for KFC
    }

    // Please not that I do not use else because both Keywords can be found in a tweet
};

stream.StartStreamMatchingAllConditions();
Linvi
Aug 11, 2014 at 5:30 PM
Hi,

Thanks for reply and code.

My code is similar to what you send, only its dynamic for keywords. Any idea/thought on how to improve performance or is there any other way to get tweets faster for said tracks or keywords.


Thanks
Shashank
Coordinator
Aug 14, 2014 at 1:26 PM
I am not sure what you are willing to do.
The FilteredStream will get all the Tweets matching your filters from Twitter, you cannot get any more than this.

As for the dynamic keywords, you will need to stop the stream, update your keywords and then restart the stream.

Linvi