Problem opening solution

Sep 17, 2013 at 11:27 AM
Hi there,

I am a uni student working on my final major project. I need to open a twitter stream and search for particular keywords, updating a register every 10 seconds. I am familiar with C# however have no knowledge of Twitter development so searched for existing APIs and found yours.

I downloaded your API however when trying to open the solution I get mapping errors.

Can you please help fix this? Also have you done searches for keywords before?

Many thanks.
Coordinator
Sep 17, 2013 at 3:44 PM
Edited Sep 17, 2013 at 5:19 PM
Hello there,

The mapping errors are 'normal'. They basically inform you that you do not have access to the Repository and that your changes won't have an impact to this repository.
If you continue opening the solution it should be working fine.

Expected Error :
The solution you have loaded is bound to the source control server at https://tfs.codeplex.com/tfs/tfs18, but no workspace on that server could be located.

  • If this does not solve your problem please let me know.
Can you please help fix this? Also have you done searches for keywords before?
You will be very pleased to know that today I have checked many new methods to track keywords from Twitter Stream.

In Program.cs you will find a section called stream. The stream allow you to receive up to 1% of the Tweets sent on twitter (4 millions/day).
Today I added features that allow you to get more information from the Twitter official filtered stream but also to filter the main stream (which receives all the tweets randomly on twitter).

Here is an example on how to filter the main stream of twitter :
SimpleStream stream = new SimpleStream("https://stream.twitter.com/1.1/statuses/sample.json");
// Starting the stream by specifying credentials thanks to the Token
stream.AddTrack("syria");
stream.AddTrack("usa", tweetMessage =>
{
    Console.WriteLine("They talk about USA in : {0}", tweetMessage);
});
stream.AddTrack("france", tweetMessage =>
{
    Console.WriteLine("They talk about france in : {0}", tweetMessage);
});
stream.StartStream(token, (tweet, tracks) => ProcessFilteredTweet(tweet, tracks));
I believe this changes will be very helpful for you.
Also if you have specific requirement that should add value to Tweetinvi I will be happy to implement it in the API.

Please send some feedback concerning these features if you use them as I would be happy to improve them based on user usages.

Sincerely,
Linvi
Sep 30, 2013 at 8:28 AM
Hi Linvi,

Thank you for responding.

I am currently trying to work out how to start the stream for 10 seconds, stop for 10 seconds etc etc..

What I am trying to do is open the stream and write all tweets to a file for 10 seconds. Then stop the stream, close the file so another process can use it and repeat.

Do you think this is possible? Have you done it before? Would it be something you would be interested in adding?

So far your API has been fantastic and I have learnt heaps.

Many thanks again.
Coordinator
Sep 30, 2013 at 10:58 AM
Edited Sep 30, 2013 at 11:11 AM
Hi,

Yes it is possible and quite easy to implement, I will send you a code example today probably.

Also if you wish to share more information concerning your project I might be able to give you some advice as I personally did University Research related with Twitter on social behavior, crowd-sourcing and social trends.

If you are willing to analyze large quantity of tweets I would recommend to not store the tweets into a file but in a database.
  • Using batch processing for the insert (inserting multiple tweets in the database at once -- I recommend batch processing > 150 tweets), you will be able to insert the tweets as fast as you receive them from Twitter. You can have a separate process that will analyze your tweets later on.
  • If you are willing to analyze your tweets in "live", insert them into a Queue<ITweet> and analyze them one by one in a separate thread that would currently have the algorithm you currently use to analyze your file.
I will send you the code example soon but please consider the alternatives that will most probably give you more accurate results for your research.

Regards,
Linvi
Marked as answer by linvi on 1/16/2014 at 2:31 AM
Coordinator
Sep 30, 2013 at 11:47 AM
Edited Sep 30, 2013 at 12:16 PM

Please read the previous comment before reading this solution!

After reading my first comment, if you still believe that alternating read/write every 10 seconds is the right solution for your problem here is a code sample for you :

Timered Stream Function :
private static void StartTimeredStream(IToken token, int timerDuration)
{
    IStream<ITweet> stream = new SimpleStream("https://stream.twitter.com/1.1/statuses/sample.json");

    EventHandler pauseStream = (sender, args) =>
    {
        Timer timer = new Timer(timerDuration);
        timer.Elapsed += (o, eventArgs) =>
        {
            timer.Stop();
            timer.Dispose();
            stream.PauseStream();
            Console.WriteLine("Stream paused for {0}ms...", timerDuration);
            // Read the file and perform the analysis algorithm
        };

        timer.Start();
    };

    EventHandler resumeStream = (sender, args) =>
    {
        Timer timer = new Timer(timerDuration);
        timer.Elapsed += (o, eventArgs) =>
        {
            timer.Stop();
            timer.Dispose();
            stream.ResumeStream();
            Console.WriteLine("Stream resumed for {0}ms...", timerDuration);
        };

        timer.Start();
    };

    stream.StreamStarted += pauseStream;
    stream.StreamResumed += pauseStream;
    stream.StreamPaused += resumeStream;

    stream.StartStream(token, tweet =>
    {
        // Write the tweet in the file!
        Console.WriteLine(tweet.Text);
    });
}
Call the function like this :
StartTimeredStream(token, 10000);
I will add this code sample in the Examplinvi at my next check-in.

Please let me know if this solves your problem (a simple yes is enough ;)

EDIT : Example available in changeset 31979

Regards,
Linvi
Marked as answer by linvi on 1/16/2014 at 2:31 AM
Oct 1, 2013 at 6:52 AM
Hi Linvi,

Thank you so much.

Live analysis is exactly what I wanted to do however with my limited knowledge of programming I was unsure of how to implement it so went for writing to a file option.

I would be happy to discuss my project further with you. Are we able to talk directly (i.e. through email?).

Thank you again for all of your help.

Kate
Coordinator
Oct 1, 2013 at 9:27 AM
You are welcome :)

"I would be happy to discuss my project further with you. Are we able to talk directly (i.e. through email?)."

Surely, you can send me an email through the contact page on my profile : linvi

Regards,
Linvi