Newsemble
API for fetching data from news websites.
Newsemble is an API that provides easy access to the current news for programmatic analysis. It has been built using Python, BeautifulSoup and MongoDB.
The data is scraped from these news websites every hour, stored in a database on the cloud and whenever requested, the most recent articles are promptly served.
Developers can make use of this API to fetch current data with each article having the following fields:
Headlines, Content, Source, Link and Time.
Technologies
Newsemble is created with:
- Python 3
- Flask
- PyMongo
- BeautifulSoup
File Structure and Description
- app.py - Flask code for the API
- scraper.py - Collection of scrapers for the various news sites.
- db.py - Connecting and Using MongoDB
- utils.py - Utility Functions
- scheduler.py - Scheduler
- Procfile - For Deployment
- requirements.txt - Python Requirments
Pipeline
