TwitterMiner

Description

You specify hashtags or users and, optionally, language, and the TwitterMiner module scours twitter's API and saves the matching tweets.

The module defines a new table in your Drupal database named 'twitterminer_tweets' and stores all of the tweets in it.

The point of mining tweets is to make them accessible for long term data analysis. Twitter only makes tweets accessible for one week through their API.

Viewing the Tweets

At the moment, the TwitterMiner module does not display tweets, or facilitate analysis. This module just grabs tweets and puts them in your database. That's it.

For now, the mined tweets can be made accessible to views using the table wizard module.

I have some analysis tools I plan on releasing soon.

This is the very start of the documentation on viewing the tweets. At the moment, all you get is a list of the modules I am using:

• Views
• Table Wizard
• Charts and Graphs
• Views Charts (views_charts-6.x-1.x-dev)

Alternative Projects

• Twitter module logs into Twitter and displays tweets.
• Twitter Block displays tweets, but doesn't log into Twitter.
• Twitter Eparser is an older mining module, using an older API. Twitter Eparser stores tweets as nodes, which requires a lot more overhead than TwitterMiner, but might be just what you are looking for if you want your tweets as nodes.

Details

TwitterMiner runs as part of cron.

The table is called 'twitterminer_tweet' and contains the following fields:

• tmtid (auto incrementing serial number)
• tmsid (the ID of the search used to mine the tweet)
• from_user_id_str
• profile_image_url
• created_at
• from_user
• id_str
• metadata (currently stored serialized)
• to_user_id
• text
• from_user_id
• geo (currently stored serialized)
• iso_language_code (based on ISO 639-1)
• id
• to_user_id_str
• source

You probably want to set up cron to run every few minutes (or less) if having a lot of data is important to your research. For hobbyists, please don't run cron frequently, or you are just burning coal and wasting bandwidth for no good reason.

Important: This module uses a lot of resources if you mine popular hashtags. If you are on shared hosting, and use TwitterMiner to mine popular tags, your shared hosting provider will probably cancel your hosting account.

Video of original (non-Drupal) version

http://vimeo.com/20178532

I had nothing to do with the software linked to above; the R-Shief project and Christopher Morton of Left Sideways developed it.

Credits

The specific inspiration for this module was the R-Shief project and their recent work analyzing tweets from the Arab revolutions.

The adorable logo was designed by Brian Beun. Brian created 5 alternatives, and the winning logo was selected by votes in the Drupal IRC channel.

TwitterMiner was developed by Inner File Software, which offers Drupal module development in Washington, DC, and TwitterMiners, which offers twitter data mining and semantic analysis.

No digital canaries were harmed in the making of this twitter mine.