Twitter Developer Platform: Using Twitter APIs for Academic Research
All the scripts require a config.ini file in which the keys are put. There is a task-specific readme in every folder.
You can install the required packages (tweepy and pymongo) with:
pip install -r requirements.txt
Part I – Building a Representative Dataset of a Location/Language
Use Case: When there’s the need to collect data in a specific language to build a dataset that is somewhat representative. We don’t want to track keywords. Instead, we can collect trending topics for a place and track those.
This script can be combined with cron to automatically downloads data every day. In a few days you can get a fairly rich dataset that can be used for further analysis.
This process is also described in the following paper:
Bianchi, F., Nozza, D., & Hovy, D. (2021, April).
FEEL-IT: Emotion and sentiment classification for the italian language.
In WASSA (pp. 76-83).
Part II – Quick and Dirty Storage with MongoDB
Use Case: When there’s the need to put up a quick streaming pipeline to track some keywords and to store them somewhere safe without thinking too much. MongoDB is a good solution because set up should be easy and python has a great integration with it thanks to PyMongo.
Part III – Reconstructing Conversations on Twitter
Use Case: This just showcases how to reconstruct conversations given a single Tweet Id.
Acknowledgements
I’d like to thank Debora Nozza and Dirk Hovy, that contributed to the work done for Part I and Mesut Kaya and Rishav Hada that are the principal contributors for the reconstructing conversations task (Part III).
Icons by Icon8
