Last week I wanted to scrape my Tweets for the past few days. It was very easy to do with Python. In this tutorial we will explore:
- How to install Twitter API library in Python
- How to set up Twitter API authentication
- How to Search with Twitter API
- How to Get User Tweets with Twitter API
Install Python Twitter API Library
For the purpose of this article, I am assuming you have Python installed and know how to access it from your terminal. All Macs come with Python pre-installed and it can be easily installed on Windows.
I used a library called Python Twitter which you can be installed via
pip install python-twitter.
Generate Access Token to Authenticate ( aka Login) to Twitter API
Once this is done you will need to get:
- Consumer Key (API Key)
- Consumer Secret (API Secret)
- Access Token
- Access Token Secret
You can get all 4 by heading over to https://apps.twitter.com.
Once there, sign in with your Twitter account and click on “Create New App” button.
Fill in required information (note that app name must be unique) and select “Create Your Twitter Application”.
You will be taken to your application view. There click on “Keys and Access Tokens” tab. Look for section called Token Actions and click on “Create my Access Token”. The page should refresh, and if everything went well you should see both Consumer Key/Secret and Access Token/Secret.
Search with Twitter API
Now that the boring part is done, we can do the fun stuff.
Open up your Python console, by running
python in your terminal. I highly recommend using iPython, which is a drop in replacement for Python console, but way better. You can get it via
pip install ipython and then running
ipython in your terminal.
In any case, first thing you’ll need to do in your Python terminal is to import python-twitter library via:
Next you’ll need to authenticate with Twitter via the following command, making sure to fill in the Consumer and Token information that you obtained earlier:
api = twitter.Api(consumer_key='FILL-ME-IN', consumer_secret='FILL-ME-IN', access_token_key='FILL-ME-IN', access_token_secret='FILL-ME-IN')
You can check that you’ve authenticated by running:
Finally the search can be performed via:
search = api.GetSearch("happy") # Replace happy with your search for tweet in search: print(tweet.id, tweet.text)
Get User Tweets with Twitter API
python-twitter library has all kinds of helpful methods, which can be seen via
help(api). All user tweets are fetched via
GetUserTimeline call, you can see all available options via:
Note: If you are using iPython you can simply type in
api. and hit tab to get all of the suggestions. Or write
api.GetUserTimeline? and hit enter to see all of the supported parameters`
In my case, I wanted to get the last 10 Tweets for myself. My Twitter username is akras14 (wink wink), so I ran the following:
t = api.GetUserTimeline(screen_name="akras14", count=10)
The following command uses list comprehension which is just a hipster way of doing a for loop on every Tweet, converting it to a Dictionary via built in “AsDict” method, and storing all of the converted Tweets into a List.
tweets = [i.AsDict() for i in t]
It’s as simple as that. Now all of the Tweets can be printed out for inspection:
for t in tweets: print(t['id'], t['text'])