Get Historical Twitter Data Using Twitter4j Libraries In Java

Get historical Twitter data using Twitter4j libraries in java

Pre-requistes :
1. Valid twitter account
2. Some knowledge of core Java.
3. Any IDE for running Java programs and knowledge of using them. (Eclipse or InteliJ)
This blog covers the following steps for getting twitter historical data :
1. Create app in twitter.
2. Download Twitter4j libraries (Jar files).
3. Write a program in Java to get historical twitter data using particular query string.

Steps are as follows :

• • Step 1: Creating App in twitter :
    1. Visit the twitter developer’s site.
    2. Sign-in with twitter account.
    3. Go to “My Applications” area and click on “Create a new application”.
    4. Fill in the application details. You can ignore the “Callback URL” field but you have to provide your website URL.
    5. After you have filled the form and submitted it, you will be directed to a page which has option to generate tokens.
    6. Click on “Create Your Access Tokens” button and choose your preferences.
    7. Make a note of/copy the following values :
    i.Consumer Key
    ii.Consumer Secret
    iii.OAuth Access Token
    iv.OAuth Access Token Secret

Step 2: Downloading “Twitter4J” libraries :
1. Download twitter4j libraries from this site http://twitter4j.org/archive/twitter4j-4.0.4.zip
2. Extract the downloaded zip file in your PC.

Step 3: Write Java program for fetching historical data from twitter using Twitter4J libraries.

1. Create a project in your IDE, and include all the jar files you have in the twitter4J zip file.

2. Java Code  :

Main Class : TwitterMain

package com.helical.twitter;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.HashMap;

import twitter4j.Twitter;

public class TwitterMain {
	private static String AccessToken = "8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxo";
	private static String AccessSecret = "sxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxC";
	private static String ConsumerKey = "Uxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
	private static String ConsumerSecret = "rxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";

	private static HashMap authTokenMap = new HashMap();

	public static void main(String[] args) {
		// TODO Auto-generated method stub
		authTokenMap.put("AccessToken", AccessToken);
		authTokenMap.put("AccessSecret", AccessSecret);
		authTokenMap.put("ConsumerKey", ConsumerKey);
		authTokenMap.put("ConsumerSecret", ConsumerSecret);

		TwitterAuth twitterAuth = new TwitterAuth();
		TwitterData twitterData = new TwitterData();
		Twitter authObject = null;
		
		String topic = args[0];
		ArrayList tweets = new ArrayList();
		try {
			authObject = twitterAuth.authenticate(authTokenMap);
		} catch (Exception e) {
			// TODO: handle exception
			System.out.println(e.getMessage());
		}
		try {
			tweets = twitterData.getTwitterData(authObject, topic);
		} catch (Exception e) {
			// TODO: handle exception
			System.out.println(e.getMessage());
		}
		System.out.println("Tweets related to " + topic + " : " + tweets);

	}

}

Authentication class : TwitterAuth

package com.helical.twitter;

import java.util.HashMap;

import twitter4j.Twitter;
import twitter4j.TwitterFactory;
import twitter4j.conf.ConfigurationBuilder;

public class TwitterAuth {
	public static Twitter authenticate(HashMap<String, String> authTokenMap) {

		ConfigurationBuilder cb = new ConfigurationBuilder();
		cb.setDebugEnabled(true);
		cb.setJSONStoreEnabled(true);
		cb.setOAuthConsumerKey(authTokenMap.get("ConsumerKey"));
		cb.setOAuthConsumerSecret(authTokenMap.get("ConsumerSecret"));
		cb.setOAuthAccessToken(authTokenMap.get("AccessToken"));
		cb.setOAuthAccessTokenSecret(authTokenMap.get("AccessSecret"));
		cb.setHttpConnectionTimeout(100000);

		Twitter twitter = new TwitterFactory(cb.build()).getInstance();
		return twitter;

	}

}

Fetch twitter data class : TwitterData

package com.helical.twitter;

import java.util.ArrayList;
import java.util.List;

import twitter4j.Query;
import twitter4j.QueryResult;
import twitter4j.Status;
import twitter4j.Twitter;

public class TwitterData {

	public static ArrayList getTwitterData(Twitter authObject, String topic) {
		System.out.println("Inside tweet data method: "+authObject + " , "+topic);
		ArrayList tweetList = new ArrayList();
		try {
			Query query = new Query(topic);
			QueryResult result = null;
			do {
				result = authObject.search(query);
				List tweets = result.getTweets();
				for (Status tweet : tweets) {
					tweetList.add(tweet.getText());
				}
			} while (result.hasNext() == true);
		} catch (Exception e) {
			// TODO: handle exception
			e.printStackTrace();
			System.out.println("Failed to search tweets: " + e.getMessage());
		}
		System.out.println(tweetList);
		return tweetList;
	}
}

3. The program takes the search topic in the form of command line argument.After providing command line argument, this code will use twitter4J methods to fetch data related to the topic entered, and give output in JSON format.

Feel free to play with the code and please comment if you find any issues with the code.
So, that’s it.. go knock your self off…!!!!

Best Open Source Business Intelligence Software Helical Insight is Here

A Business Intelligence Framework

Free
Download