Skip to content

A collection of web scraping projects to practice your skills or build a portfolio

Notifications You must be signed in to change notification settings

israel-dryer/Web-Scraping-Projects

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 

Repository files navigation

Web-Scraping-Projects

A collection of web scraping projects to practice your skills or build a portfolio.

Project Description
Amazon product and price data
Indeed.com job postings based on search criteria
Yahoo! Finance financial, company and historical stock data
Salary.com salary statistics based on specific search criteria
Yahoo! News news article data including summmary based on search criteria
Twitter scrape twitter data
Ebay scrape ebay searches

Requirements

A few of these projects use browser automation, but most do not. For this, I use Selenium to automate the browser. Other libraries that are used include:

  • Requests
  • BeautifulSoup
  • lxml

Expectations

These projects are designed to give you experience web scraping, but assume that you have some basic familiarity with at least Requests and BeautifulSoup. Selenium is not used extensively enough to need familiarity, but you will need to install it on the few projects that require it.

Caution

While I will try to keep these projects updated, please keep in mind that websites can change at any time, rendering an existing scraper useless. This is unfortunately the nature of webscraping. Your production models will require constant attention and maintenance to ensure they are delivering the data and results that you expect.

About

A collection of web scraping projects to practice your skills or build a portfolio

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published