Web Scraping Introduction Python Selenium Splash Scrapy

Web scraping from different sources using various libraries and frameworks

What you’ll learn
Web scraping using different Python libraries
Getting page’s HTML using requests and beautifulsoup4
Scraping Multi-Page Sites Using Scrapy
Website authorization and data collection using Selenium

Requirements
Basic Python knowledge
Knowledge of HTML markup

Description
This course is designed for those who want to learn how to automatically collect information from various sites, that is, scraping. Basic knowledge of Python is required to complete the course.

The lectures will cover the basic approaches to web scraping using libraries such as requests, beautifulsoup4 and Splash, as well as the Scrapy and Selenium frameworks.

This course covers the basic approaches to web scraping. At the end of the course, students will have a basic understanding of how to scrape various sites using Python.

This course will not be very interesting for those who already have a basic understanding of what web scraping is and how to create scripts for automated data collection.

After completing the course, students will know

How to find and install the required libraries for parsing data.

What is ethical parsing, how not to violate the rights and the law when scraping data.

What is the robots.txt file for?

How to get the HTML of the site to extract the necessary information.

How to use xpath to specify the exact location of elements on a page.

How to navigate through pages on a multi-page site.

How to create a project and run Scrapy spiders.

How to scrape all information about each product on the site.

How to navigate to the page of each product.

How to scrape sites that use JavaScript on load.

How to log in to the site using the Selenium framework.