How to convert indirect URL to direct one.

May 7, 2014 at 5:35 PM
I use a filteredstream with a number of filters. When a tweet event fires, I need to deduce which filter the tweet relates to, as this affects what I do with the information.

This is fine, except in some cases the filter has been matched not on the text in the tweet, but on a URL which is embedded in the tweet, unfortunately the URL presented in the tweet is an indirect shortened one which contains no trace of the original filter. For example, using a filter of the word 'golden' I receive this tweet:

[Raspians (@Raspians) posted] RT @TheRascalPower: @Raspians https://t.co/nZLJdBm9bI help us be the change! #BeTheChange #charity #india #charitywater #Travel #WeNeedYo…

No mention of the word 'golden' in there. But if browse to the short URL it redirects to
https://www.indiegogo.com/projects/documenting-change-the-golden-triangle#team&ui-state=dialog

which does contain the word 'golden'.

Is there any way I can get the hold of the original URLs from the Tweet object?

Thanks.
Coordinator
May 8, 2014 at 12:03 AM
Hi,

Normally you should be able to access this information from the Entities.
ITweet.Entities.Urls[0].ExpandedURL
Linvi
May 8, 2014 at 7:42 AM
Many thanks linvi. I missed that one.
May 8, 2014 at 8:50 AM
Edited May 8, 2014 at 9:07 AM
Actually that doesn't seem to do it.

A tweet (ID 464324953815269376) has text of
"Forget 1960, the golden age of air travel is now http://t.co/if2l5TqG4g"

URLs collection has a count of one, and its properties are as follows
    DisplayedURL    "bit.ly/1uEvVp7"    
    ExpandedURL "http://bit.ly/1uEvVp7" 
    URL                 "http://t.co/if2l5TqG4g"    
If I browse to http://t.co/if2l5TqG4g, it redirects to :
http://economictimes.indiatimes.com/news/international/business/superjumbo-jet-haute-cuisine-vintage-wine-this-is-the-golden-age-to-travel-by-air/articleshow/34824700.cms?utm_medium=twitter&utm_source=twitterfeed

which does not appear in the collection.

Any ideas?
Coordinator
May 8, 2014 at 10:01 AM
Hi,

I think the information are correct. The issue is that there are 2 redirection instead of a simple one as you would expect.

The user posted the tweet with "Forget 1960, the golden age of air travel is now http://bit.ly/1uEvVp7".
Twitter transformed this tweet into ""Forget 1960, the golden age of air travel is now http://t.co/if2l5TqG4g""
This URL redirects to the indiatimes URL (the big one).

And twitter created a new short URL (http://t.co/if2l5TqG4g) for http://bit.ly/1uEvVp7.
It means that when you click on the link the following happens.
  1. Go To http://t.co/if2l5TqG4g
  2. Twitter.com Redirects To http://bit.ly/1uEvVp7
  3. Bit.Ly Redirects To http://economictimes.indiatimes.com/news/international...
Consequently the data retrieved are correct.
Did you retrieve this tweet from the FilteredStream?

Regards,
Linvi
May 8, 2014 at 11:14 AM
Yes this was recovered from FilteredStream

I think it may be OK. The last example I gave did not have an expanded URL, but then again it matched on the Tweet text anyway.

I have run a few more examples through and I can see that it the expanded URL is being provided as expected and is being matched by the filteredstream.

Thanks for your help.