A list of tools for finding or capturing tweets

Last updated: 13 October 2012
Shortened link for this post is http://is.gd/ennsBC

3 November 2012: Interim edits (cos I'm not doing this in one go!) - I'm at twapperlyzer in the list below. Anything below that hasn't been checked for a while, everything above has been.
 
Post history: This post was born on 30 May 2011 at a time when there were many more free tools available with which to wrangle tweets. Since then Twitter's changed its terms of service and Google Realtime no longer exists making some of the information now wildly out of date, in October 2012. The original post will be placed HERE as a PDF for completists and archivists who want to know how things used to be. But the text below has been significantly changed.

I've found Quora to be the best resource for finding out about new tools for a couple of reasons: (1) is that Quora answers persist and don't fall off the edge of the page compared to Twitter and (2) people organise the information into questions and answers making it much easier to find! Also people tend to post this info to Quora more than on Twitter (or I'm not following those people). 

What have we got here then?

Part A is the alphabetic list, with a bit more info about each individual tool.

Part B looks at a range of tools according to the function(s) they can perform - B1 for finding tweets, B2 for trapping tweets and B3 for handling conference or event hashtags.

I might have sucked all the fun out of this post by itemising everything but it was getting a bit chaotic ;)

Currently I use SearchHash, Epilogger, Chirpstory, Storify and / or Twilert to trap hashtagged event tweets. I’m also using Storify for small events because the interface is so good and it lets you contextualise things. But Chirpstory is very good as a quick trapping tool too, for larger numbers of tweets - last time I looked you can grab 20 tweets at a time with Storify and 50 at a time with Chirpstory.

Part A: alphabetic listing of tools 

• BackTweets – "search for links on Twitter"
  I searched for brodiesnotes.blogspot.com and found quite a few tweets including a couple from 2009. The search results don't actually give you either the link to an individual blog post (only the main blog homepage address) nor do they give you the link to the original tweet, though that is discoverable.
• Bing site search – assume same as Google, not tried.
• Bing social – Bing’s Twitter search
• Blackbird Pie – lets you embed an example tweet in a blogpost - my example here. Defunct as Twitter's now makes it possible to embed tweets in blogs very well. To get the embed code click on any tweet to expand it, then click on Details, then grab the code.
• Chirpstory.com – suggested by @annindk. Similar to Storify.
  This letss you grab a bunch of tweets (up to 50 at one go) and also lets you reorder them with one click so that readers see the earliest tweet first. Very useful tool, one of my favourites. You can also easily remove RTs and any duplicates you might have copied over twice. Here's one I made earlier http://chirpstory.com/li/1600
• Curated.by – no longer functions, kept in list for completeness.
• Embedding a tweet somewhere - see the information above in Blackbird Pie, it's easy to do now.
• Digestly - lets you receive, by email, tweets from any user with a public account
• Epilogger – brilliant. This automates the saving of hashtagged tweets and can pick up tweets earlier than the time at which you set up your 'log'. Currently can't embed into blogs, but does provide a basic analysis of who's tweeted the most on a hashtag.
• Exquisite Tweets – lets you thread a bunch of tweets together, similar to Chirpstory and Storify, h/t @tomscott
• Favotter – no longer functions, kept in list for completeness.
• Favstar – lets you see favourite tweets (yours, or someone else’s) – since not every single one of someone’s tweets will be favourite this is a highly selective filter of available tweets. Here are MY tweets that OTHERS have retweeted.
• Freezepage – this used to trap an exact copy of whatever web page you were looking at in case the page is changed or deleted in future, however I've noticed it doesn't work as well for Twitter and have stopped using it. Here's one I made on 13 October. My favourite example of frozen tweets marks Gillian McKeith's rudeness to someone pointing out she has no real PhD qualification.
• FriendFeed – n 2008 I imported my Twitter feed (http://twitter.com/JoBrodie) into this tool and it's still capturing them. I thought it would stop what with Twitter mucking about with what it lets escape from its 'ecosystem' but so far all is well. Here's my first tweet, which I originally found in FFeed but time has passed and it doesn't appear by searching anymore.
• Google Advanced Search – well, always worth a go I suppose.
• Google Realtime – formerly awesome, now no longer with us.
• Google site-restricted search – use site:twitter.com to search within Twitter
• Google Spreadsheets – "A hack for searching for tweets by date, with help from Google Spreadsheets" by @steveWINton and highlighted to me by @CliveAndrews - I have no idea if this will work still, it looks pretty technical. If the Twitter export doesn't work, try RSS feeds until Twitter stops supporting them too in March 2013.
• GrabChat.com – made by @dean_jenkins for people to 'archive your conversation'. It lets you export a whole bunch of hashtagged event tweets, really good.
• Guardian Twitterfall – app for events where you want to pre-moderate the tweets that appear on the wall – it's for people who understand what JSON and AJAX mean, sadly that doesn't include me.
• hashtags.org – apparently it's meant to show you tweets with a particular hashtag along with a little graph but I think you have to spend money for it to do so, so I've not bothered with it – via this slideshare presentation.
• Hashtracking - via rjleaman. I follow them on Twitter (@hashtracking) and often pick up some really interesting stuff, although I've not really tried out the tool very much. I tend to go for the cheapy freebies but I hear good things about this one, probably best for corporate use if you're a cheapskate like me.
• Hashtweeps - type in a hashtag and the service will tell you people who've been tweeting with it - via this slideshare presentation.
• Healthcare Hashtags - if you have a health-related hashtag you can add it and capture tweets here. Not tried it for non-healthcare purposes.
• identi.ca – an alternative to Twitter, on signing up you agree to your tweets being open – wondered if enough people were using this to make it worthwhile using in a conference setting.
• IFTTT – If This Then That.
  An excellent tool for all sorts of social media stuff. Twitter triggers have now been disabled at Twitter's request so you'll need to use RSS as the trigger instead. That will stop working after March 2013  less so for Twitter.
  >> Here are the suggested Twitter RSS feeds you can use.
  >> More info on the changes to IFTT here or here 
• Keepstream.com – it's read only now.
• Library of Congress – they keep every tweet but it's not available to the public, only to a small number of researchers. No idea if this will change in the future.
• Monitter - lets you watch a stream of tweets in real time. See also TweetChat and Twitterfall.
• PeopleBrowsr- lets you scroll back through several hours' worth of a person's tweets or tagged tweets. You get 10 free goes. Not really sure how it's unique.
• Research.ly – appears to have turned into PeopleBrowsr
• Roundteam - automatic retweet service [added 28 Feb, not investigated]
• Rowfeeder – 500 hashtagged tweets a month free, but more than that is chargeable on a sliding scale, via this Quora post
• RSS – Twitter will stop supporting RSS after 5 March 2013 which is a bit of a shame. If you're using IFTTT and your Twitter triggers aren't working, just edit your recipes so that the trigger is an RSS version of the same Twitter feed.
  
  For an individual user:
  https://api.twitter.com/1/statuses/user_timeline.rss?screen_name=jobrodie
  
  For a person search:
  http://search.twitter.com/search.atom?q=jobrodie - this will find all instances of people posting TO me as well as all instances of ME posting. Note that every tweet is duplicated, goodness knows why.
  
  For a person search restricted just to @mentions:http://search.twitter.com/search.atom?q=@jobrodie
  
  For your own favourites usehttp://api.twitter.com/1/favorites.rss?screen_name=jobrodie
  
  For a keyword search:http://search.twitter.com/search.atom?q=fluffy%20bunnies (%20 acts as 'space')
  
  For a hashtag search:http://search.twitter.com/search.atom?q=%23scicomm  (%23 acts as '#')
  
  If you see the %20 and %23 replacing [space] and # respectively and are thinking "what else can I find to play with" you might like http://en.wikipedia.org/wiki/Percent-encoding
  
  
• SearchHash - wonderful. Does almost exactly what WTHASHTAG used to do - lets you archive and export a pretty complete record of a hashtag.
  
  You are also given the tweet ID and the 'from user' author in the table. To find the link for the original tweet use the formula http://twitter.com/AUTHOR/status/TWITTERid eg https://twitter.com/JoBrodie/status/75299030430777344 You can export tweets as a .csv but the data table isn’t the prettiest of things to share with people so Twilert might suit better.
• Searchtastic – shut down as they didn't have time to maintain it
• SearchTweetsByDate – appears to be dead
• SnapBird – lets you search a little further back than Twitter's search (though Storify or Chirpstory do this and I use them more)
• Social Searching – no idea, log in with Facebook - can anyone tell me what it does?
• Storify – I use this a lot but not quite a much as Chirpstory. Once you've trapped a tweet in a Storify story it will remain there even after the original tweet has been deleted. Use this power wisely...
• tagdef – find a hashtag and see what it means, or define it yourself if you know - via this slideshare presentation. It's not possible to 'reserve' a hashtag and say 'it means this' and others can't use it, but sometimes it's helpful to know what it's supposed to mean. See also tagalus.
• tagalus or tagal.us – was similar to tagdef, now permanently down. From their Twitter account it appears the last tweet was sent in November - via this slideshare presentation.
• The Archivist – I found this quite good but you have to keep an eye on the tweets as they disappear ‘off the bottom’ – very easy to miss tweets if you don’t actively monitor it. You can download in .csv (spreadsheet) or zipped file formats. Desktop version also available.
• ThinkUp – downloadable free open-source app which analyses your tweets, Fb and Google_ activity. If you have your own server you can use it on that, if not they've partnered with a cloud company PHP Fog and you can host it there instead. h/t @martinjmckenna
• Topsy – search the social web: this is very good and can find old tweets (I found some from 2009) but it doesn't let you scroll back and forward like Google Realtime used to let you.
• Twapperkeeper - closed on 6 January 2012 - before that it let you access two archives free, and you had to pay for more than two archives. See also YourTwapperkeeper, the open source version - its current status is unknown to me. You can also query (but not create) Twapperkeeper archives using Eduserv's Summarizr (h/t @briankelly)
• Twapperlyzer - seems to be linked to YourTwapperkeeper (see Twapperkeeper) h/t this slideshare presentation.
  Point at which I stopped editing... info below may be out of date
• Twazzup - looks like a news story thing, akin to paper.li, not investigated in depth yet as it's currently reporting a 500 error, h/t @davoloid.
• TweetBoard - unavailable
• TweetChat - really good for watching a live stream. You can type in your hashatag of interest and it will present you with all the tweets in real time. If you miss a few you can scroll back and it will pause for you. Used very successfully in the first Twitter Journal Club aka #twitjc. You can also sign in via Twitter authorisation and tweet directly from the programme (it will automatically add the relevant hashtag to all your tweets). It also lets you block spammers from view (hat tip @fidouglas and @silv24). See also Monitter and Twitterfall.
• Tweetdoc - lets you create a PDF document bringing together not more than 500 hashtagged tweets and keep them as a permanent record h/t @martinrue
• TweetGrid - automatically updates a hashtag, so similar to things like Monitter or Twitterfall, try the search facility and you can add a Twitter search widget to your page - h/t this slideshare presentation.
• Tweetnotes - stalled early in development and is no more, sadly, as it showed promise.
• Tweetreach - calculates how far your tweet travelled (I assume it counts up the maximum number of eyeballs that could have seen it based on followers)
• TweetReports - ££
• TweetScan - temporarily closed to new registrations so I can't tell you anything about it
• Tweet Tunnel - lets you go back 3,200 tweets (as Twitter itself does) but you don't have to scroll through all of them but can 'page' through (random access versus linear access I suppose!)
• Twitterfountain - Currently inactive though website promises a revamp. Was a wonderful way to present tweets at conferences on a display screen. Until it's back, use Twitter Fontana etc.
• Twitter Fontana - pretty cool, for use in foyer display screens where you want to present tweets prettily. No use for archiving though.
• Twitter Search - Twitter's own (much improved) search facility, note that there's also an Advanced Search but it's still fairly limited (note, it's very good it's just that I compare everything to the no-longer-with-us awesomeness of Google Realtime).
• Twitter widgets - "Widgets let you display Twitter updates on your website or social network page" - if you want to show, on your webpage, what you're sharing on Twitter then use one of these widgets. There's an example on the linked page. See also B3.3 below.
• Twilert - I like this a lot. It's good for capturing event hashtags by emailing them to you and the appearance is nice too (not sure how you can display tweets sent to you by email though). It sends in batches of 100 and you can have them sent every 15 mins if it's a very busy event. I wrote about it here. It's been around since 2009 and I only heard it about it at the end of June beginning of July 2011!
• TwimeMachine – will let you access the last 3,200 tweets in a person's timeline
• Twitter Advanced Search - I'm personally not a fan as it doesn't offer me any advantage over Google Realtime, but it is an exceptionally easy user interface (OK it is easier than GRealtime, but you can't scroll back and forwards in it which you can on Google).
• Twitterfall - lets you watch a stream of tweets in real time. See also TweetChat and Monitter.
• Twitter itself: the ‘More…’ scrollback button at the bottom lets you access 160 pages of 20 tweets per page (3,200 tweets) https://twitter.com/JoBrodie?max_id=60838663235452928
• twittertoirc - "Twitter-to-IRC bridge: A hacky Ruby script to route Twitter status updates into IRC." by @tommorrisI said in an earlier edition of this post: Another tool I have ‘mentally’ investigated is IRC (internet relay chat) as a backchannel for conferences. I’m sure all nerdy conferences have an IRC channel but for regular conferences possibly not – these let everyone sign in, all posts are labelled as who wrote them and it’s possible to create a transcript. Seems perfect – what I’d like to do, but am not clever enough to work out how, is to export a live Twitter RSS of a hashtag into an IRC channel and capture the tweets that way (maintaining tweets’ authors identities). – then discovered Tom had written this programme.
• Twitter tools - Wordpress plugin, see How to auto publish your Twitter tweets as Wordpress posts and the Twitter tools plugin FAQ.
• Twoogel
• twtr2rsc beta - ostensibly sends you tweets by email (as Twilert does) or to Evernote (as IFTTT and a Twitter RSS feed trigger does, until March 2013) but I couldn't get anywhere with it.
• UgoTwitt - your Twitter replies sent by email. May not work now Twitter's tweaked things. 
• Visible Tweets - lovely, lets you display hashtagged event tweets in a nice way on a large screen in the foyer at your event or in the event room h/t @clangyandjammy. If you want people in the room to see tweets as they're coming in though I'd recommend something like Twitterfall or Monitter.
• Wayback Machine - entirely random, very limited.http://classic-web.archive.org/web/*/http:/twitter.com/JoBrodie
• WordPress plugin - trap de novo, curate from this point forwards - see Twitter tools
• wthashtag - What the Hashtag, no longer functioning in the way it used to alas and is now What the Trend API. If you have programming skills (JSON...?) you may get something useful from it but it's beyond me.
• Yahoo pipes – haven’t the faintest idea how to get one to work, but clever people might play around with this and send me some instructions as it seems to be potentially useful. Some info at http://thecodetrain.co.uk/2009/05/following-twitter-conversations-using-yahoo-pipes/
• Yahoo site search – not investigated but assume it will be similar to Google’s
• Yahoo twitter search – I think this is similar to Google’s Realtime search
• YourTwapperKeeper – open source version of TwapperKeeper

Part B: tools ordered by functionB1. Finding tweets either from May 2010 onwards or 3,200 tweets ago and no earlier
Note that all of your tweets still exist somewhere on Twitter's servers but they stop being amenable to its searches once they are superseded by a whole bunch of other tweets. 

To prove that old tweets don't really disappear here's Jack Dorsey's tweet from 22 March 2006 https://twitter.com/jack/statuses/81 - he co-founded Twitter I think. This might be the earliest tweet that remains on Twitter.

B1.1 Scrolling back on Twitter
If you've posted fewer than 3,200 tweets in total (including RTs) then you will be able to scroll continually back (160 pages maximum at 20 tweets per page) to your first one. On New Twitter the page will continually expand with another 20 tweets as you scroll to the end of the page, but on old Twitter you'll need to click on the More button.

B1.2 Fast scroll (jump) by tweaking a web address: 'URL hacking' - old Twitter onlyYou should also be able to find your own Twitter URL that will let you jump back pages by changing numbers in the URL. It seems that this might not work if you're using new Twitter and it's pretty fiddly anyway. Scroll to the bottom of the page until you see a More button - this might not appear on new Twitter - hover over it and make a note of the URL that appears in the status window (I open up a small notepad window to the side and type it in). You'd think you could right click, save URL - but no - for some reason left clicking and right clicking both activate the More button so you will have to make a note manually (you could also take a screen shot of the page using Print Screen, copy it into Paint or Word and take down the details at leisure).

Once you have the URL (eg something like this) https://twitter.com/jobrodie?max_id=86927876561190912&page=2&twttr=true - annoyingly the numbers bit changes so I don't think you can just make a note of this and use the same one. Paste it into a new tab or window and change the bit saying page=2 to page=20 and so on. The max is page=160.

- arrows point to the URL you need.

B1.3 Find your first tweet
Posted more than 3,200 tweets? Then it's probably not straightforward.
If it's working (and you have fewer than 3,200) you can cheat and use http://myfirsttweet.com/, or the 'jump' function mentioned above. If you've been a bit chattier and have several thousand then you'll have some difficulties finding your first.

Google Realtime is a wonderful thing from Google that lets you find tweets going back as far as May 2010 - I use it quite regularly, more details below in the list of tools.

It was my good fortune shortly after signing up to Twitter to import all of my tweets into my FriendFeed account meaning that I could search, very easily, for any of my tweets. FriendFeed's search has become much less reliable of late - but it did (though no longer does) let me find my very first tweet https://twitter.com/JoBrodie/statuses/841825864. Oddly, searching now for this no longer works - instead of picking up the original tweet, it picked up only the later search I ran for the tweet in 2009 ;)

B1.4 Finding tweets from a week or two agoI think Google Realtime is probably the best for this, see more details and alternative in the list below. Possibly much more difficult now unless you have tweets fed into FriendFeed. You could also give Topsy a go.

B1.5 Find tweets on a topic or hashtag
Twitter search is good for finding recent ones, just type in your keyword(s) or hashtag. For older ones, use Google Realtime.

B2 Trapping tweets – see also B3.2Freezepage, Chirpstory and Storify will let you capture a tweet. It will remain captured even after deletion, see Freezepage and Storify details above.

B3 Conferences and events: for events organisers and delegates
B3. 1 Follow an event hashtag in real-time, at conference or at home
Twitter itself is pretty good for this - just search for the hashtag and sit back and watch the tweets auto-refresh. Also particularly good is Tweetchat - neither requires you to have an account to watch the tweets. If you have a Twitter account you can sign in to Tweetchat and send tweets that have the hashtag of interest already written in the tweet for you. More info below, and see also Monitter and Twitterfall.

B3.2 Display a Twitterwall in the foyer or the conference roomPrettiest so far seems to be Twitterfountain, ideal for the anteroom display for people to coo over. For a backdrop that people can follow inside the event room I prefer Twitterfall (actually I prefer no Twitterwall but…). See also Guardian Twitterfall.

B3.3 Display event tweets on your website
If you want to bring in the results of a search for an event hashtag you can do that with Twitter Widgets. The 'Search' widget (the options are Profile, Search, Favorites, List) will pull in a hashtag or other keyword but if you worry that cheeky types will introduce rude words into the tag (or just want to show a selection) then you can instead use the 'Favorites' widget and as you favourite each relevant tagged tweet it will show up. This requires more work from you though... See also TweetGrid (listed above) for an option to let people search for anything on Twitter via a widget you can add to your site.

B3.4 Saving an event hashtag for later
Best in show is SearchHash or Twilert but there are a few other tools available too. Twitter's made it a great deal harder for people to download / export tweet archives. The sooner you set these up the better (ie before the event begins) because people often use the hashtag a day or even a week / month or two before an event begins. Chirpstory is great for bagging a run of tweets soon after an event and lets you capture them a page at a time whereas the similar Storify lets you capture them individually but also lets you intersperse commentary. The first two are better for big conferences, the latter are better for small things. Previously everyone used wthashtag (What the hashtag) but it no longer works, and Twapperkeeper only lets you create two free archives.

B3.5 Format tweets for conference output documents, eg blogs Storify is lovely for this – you can embed the final Storify story into most blogs, but to place an individual tweet into a blogpost directly… I need to find out what the tool is that I’ve seen used!

Found one - an example is Blackbird Pie which lets you select a tweet and 'bake' it into your blogpost. I've linked an example of me trying it out in the alphabetic list above. Although I've not tried it I understand the Twitter widget in WordPress will also do this.

Further information on tools mentioned above
The end of Twapperkeeper? (and what to do about it) March 8, 2011, 11:00 am By Mark Samplehttp://chronicle.com/blogs/profhacker/the-end-of-twapperkeeper-and-what-to-do-about-it/31582

Top 10 Websites To Search Old Tweets By YOUNG
February 16, 2011
http://freenuts.com/top-10-websites-to-search-old-tweets/

All the easiest ways to search old tweets August 11, 2010
by DAVE LARSON
http://blog.tweetsmarter.com/twitter-search/10-ways-and-20-features-for-searching-old-tweets/ - covers some of the options listed above, and others I’ve not heard of.

7 ‘Secret’ Ways To Use Twitter Search June 25, 2009
by Thomas @Baekdal
http://www.twitip.com/7-%E2%80%98secret%E2%80%99-ways-to-use-twitter-search/

Other posts by me, on related topics

• Some quick thoughts on Storify as a mini archive for Tweets (4 May 2011)
• Screencap video that I made about Google Realtime (30 April 2011)
• Google Realtime: Finding threaded conversations on Twitter as well as year old tweets (14 April 2011) - features Google Realtime, Storify, Wayback Machine and my failed attempts to find the searchable archive at the Library of Congress
• Trapping tweets for archiving eg at conferences, or curiosity (5 March 2011)