I’ve taken some interest in URL shorteners recently. URL shorteners are the latest fad in web services – everybody wants to have their own, even if it’s not yet entirely clear how to profit from it, at least not for us mere mortals. They were born out of Twitter users need to compress their microblogging posts (some links are actually longer than 160 characters, believe it or not) and it’s spread everywhere on the Internet. Major web sites like Facebook, Google and Youtube are into it and -some say- it’s one step closer to destroying the Web as we know it.
Of course, this can also be used for evil purposes. :)
So I made myself a Python module to toy with these things. It can be used as a command line tool as well as a module to be imported in projects of your own, and contains no non-standard dependencies. You can download it from here: shorturl.py.
For example, this is how you convert a long URL into a short one (the shortest possible choice is automatically selected):
$ ./shorturl.py http://www.example.com/
And it’s reverse, converting that short URL back into the long one:
$ ./shorturl.py -l http://u.nu/63e
You can also choose a specific URL shortener service. Let’s try Twitter’s new favorite, bit.ly:
$ ./shorturl.py http://www.example.com/ -u bit.ly
RuntimeError: No data returned by URL shortener API
RuntimeError: <h3>URL has wrong format! Forgot 'http://'?</h3>
Short : http://easyuri.com/657ae
Short : http://tinyurl.com/yhabhth
Short : http://xrl.us/bgsge5
As a matter of fact, TinyURL seems to accept anything you send it, may it look like a valid URL or not. This has lead some people to build a rogue filesystem called TinyDisk, world readable, yet very hard (if not impossible) to trace back to it’s creator. The download link seems to be down but the Slashdot article is still there. I also found this article explaining how such a filesystem would work.
But… how well do they behave when you shorten the URL multiple times? First, let’s try shortening a link with TinyURL three successive times:
$ ./shorturl.py -c 3 -u tinyurl.com http://www.example.com/
The preview feature only seems to work for the first redirection (see for yourself). Now let’s try shortening our example URL three times, using random providers:
$ ./shorturl.py http://www.example.com -v -c 3
I found the Echofon plugin fails to expand this short URL. However the longurl.com service works like a charm – it even told me how many redirections I used! Click here to see it for yourself.
Just for fun, how about an infinite recursion? First, we create a link to a shortened URL that doesn’t yet exist.
$ python shorturl.py -v -c 5 http://tinyurl.com/thisisaninfiniteloop
Now let’s go to the TinyURL webpage and create the link manually. Voila! We have an infinite redirection. The longurl.com service seems to break now (see for yourself) but this is of no consequence – the link doesn’t really take you anywhere. Other apps like wget get stuck in an infinite loop however. This is probably of no real use to an attacker but I thought it was fun to try it out. :)
As a final note: while writing this I came across this article describing even more risks inherent to URL shorteners: information disclosure. A dedicated attacker could dump randomly chosen shortened URLs to retrieve information like corporate intranet links, user IDs and even passwords. I liked the idea, so I whipped out a small script to test it out:
from random import randint
from shorturl import longurl, is_short_url
characters = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'
token = ''.join([ characters[ randint(0, len(characters) - 1) ] for i in xrange(6) ])
url = longurl('http://bit.ly/%s' % token)
if not is_short_url(url):
print "%s => %s" % (token, url)
But the results of bruceforcing didn’t seem so great. Then again, one could also wonder how random this short URLs really are… maybe there’s a way to predict them, or at least reduce the number of URLs one has to try. One could also try a dictionary based approach for services like bit.ly, which provide the users the ability to pick custom short URLs. Update: Those TinyURL links weren’t so random after all ;) here’s an interesting project on URL scraping.
You can get the source code at Github.