The ‘Scrapy: Powerful Web Scraping & Crawling with Python’ course will teach you how to scrape websites and build a powerful web crawler using Scrapy, Splash and Python. The course also teaches how to Integrate Splash with Scrapy to scrape JavaScript rendered websites.

The course teaches how to create a web crawler in Scrapy and also you will learn how to deploy & schedule Spiders to ScrapingHub. The course is usually available for INR 2,599 on Udemy but you can click on the link and get the ‘Scrapy: Powerful Web Scraping & Crawling with Python’ for INR 499.

Who all can opt for this course?

  • This Scrapy tutorial is designed for individuals who are already familiar with Python and want to learn how to build a web crawler and scraper that is effective for navigating websites and extracting content from pages that contain useful information

Course Highlights

Key HighlightsDetails
Registration LinkApply Now!
PriceINR 499 (INR 2,599) 78% off
Duration10.5 Hours
Student Enrollment16,133 students
InstructorGoTrained Academy
Topics CoveredWeb Scraping, Python, Xpath, API
Course LevelAdvanced
Total Student Reviews2,726

Learning Outcomes

  • Use Scrapy to build a web crawler
  • Scraping data by crawling one or more pages
  • Spider deployment and scheduling on ScrapingHub
  • Scrapy’s use for website logins
  • Scrapy being run as a standalone script
  • Integrating Splash with Scrapy to scrape websites that use JavaScript
  • Scraping JavaScript-driven web pages with Scrapy and Selenium in Particular Situations
  • Constructing an advanced scrapy spider
  • Further features that Scrapy provides when Spider has finished scraping
  • Using Scrapy Parameters for Editing
  • Creating files in CSV, Excel, XML, or JSON from data scraped by Scrapy
  • Putting data that Scrapy has extracted into databases like MySQL and MongoDB
  • Several actual web scraping projects, including those for Craigslist, LinkedIn, and numerous more websites
  • You can download the Python source code for each exercise in this Scrapy tutorial
  • Use the Q&A board to post your inquiries and immediately receive responses

Course Content

S.No.Module (Duration)Topics
1.Scrapy vs. Other Python Web Scraping Frameworks (11 minutes)Scrapy vs. Beautiful Soup vs. Selenium
Course Tips (Must Read)
2.Scrapy Installation (10 minutes)Linux Scrapy Installation
Mac Scrapy Installation
Windows Scrapy Installation
Scrapy Installation Instructions
Python Editor: Sublime Text
3.Building Basic Spider with Scrapy (34 minutes)Scrapy Simple Spider – Part 1
Scrapy Simple Spider – Part 2
Scrapy Simple Spider – Part 3
4.XPath Syntax (08 minutes)Using XPath with Scrapy
Tools to Easily Get XPath
5.Q&A (28 seconds)Scrapy Basics
Do you have questions so far?
XPath Syntax
6.Building More Advanced Spider with Scrapy (38 minutes)Scrapy Advanced Spider – Part 1
Scrapy Advanced Spider – Part 2
Scrapy Advanced Spider – Part 3
Scrapy Advanced Spider – Part 4
Scrapy Architecture
7.Web Scraping Best Practices (01 minutes)Avoid Getting Banned!
8.Deploying & Scheduling Scrapy Spider on ScrapingHub (10 minutes)ScrapingHub: Deploying & Scheduling Scrapy Spiders (UPDATED)
9.Logging into Websites Using Scrapy (17 minutes)Logging into Websites Using Scrapy
10.Scrapy as a Standalone Script (UPDATED) (02 minutes)Scrapy as a Standalone Script (UPDATED)
11.Building Web Crawler with Scrapy (10 minutes)Building Web Crawler with Scrapy
12.Scrapy with Selenium (23 minutes)Why/When We Should Use Selenium
Selenium WebDriver + Scrapy Selector to Extract URLs
Selenium Loading Next for Data Extraction (usable even with JavaScript pages)
Getting Data
13.Scrapy with Splash – JavaScript Websites (01 hour 03 minutes)Splash Prerequisite: Install Docker (NEW)
Splash Installation (NEW)
How to use Splash with Scrapy (NEW)
Splash Advanced Project: Scraping p.1 (NEW)
Splash Advanced Project: Scraping p.2 (NEW)
Splash Advanced Project: Scraping p.3 (NEW)
14.Scrapy Spider – Bookstore (26 minutes)Grabbing URLs
Data Extraction
15.More about Scrapy (15 minutes)Scrapy Arguments
Scrapy Close Function
Scrapy Items
16.Export Output to Files (29 minutes)Scrapy Feed Exports to CSV, JSON, or XML
Export Output to Excel
Downloading Images with Scrapy Pipelines
Renaming Images with Scrapy Pipelines
17.Scrapy Project #1: Scraping Craigslist Eng Jobs in NY (51 minutes)Craigslist Scraper – Overview
Creating Scrapy Craigslist Spider
Craigslist Scrapy Spider #1 – Titles
Craigslist Scrapy Spider #2 – One Page
Craigslist Scrapy Spider #3 – Multiple Pages
Craigslist Scrapy Spider #4 – Job Descriptions
Editing Scrapy (e.g. throttling, user agent, etc.)
Final Scrapy Tutorial, Craigslist Spider Code
18.Extracting Data to Databases – MySQL & MongoDB (39 minutes)Installing MySQL
MySQL Installation and Usage
Writing Data to MySQL
Installing MongoDB
MongoDB Installation and Usage
Writing Data to MongoDB
19.Scrapy Project #2: Web Scraping (32 minutes)Scraping Class-Central – Part 1: Subjects (UPDATED)
Scraping Class-Central – Part 2: Courses (UPDATED)
20.Scrapy Advanced Topics (59 minutes)Scrapy User Agent
Scraping Tables (UPDATED)
Scraping JSON Pages
Scrapy FormRequest (UPDATED)
Using Multiple Proxies with Crawlera (Optional)
21.Scrapy Project #3: Web Scraping Dynamic Website (01 hour 01 minutes)ePlanning Scraping Project Overview
ePlanning: Extracting Initial URLs
ePlanning: Crawling Internal Pages
ePlanning: Scrapy Form Requests
ePlanning: Scraping Data
ePlanning: Checking Data Existence
ePlanning: Scraping Data from Table
22.Project #4: Scraping Shoes’ Prices from API Request (36 minutes)Scraping Product Prices from API Request p.1 (NEW)
Scraping Product Prices from API Request p.2 (NEW)
Scraping Product Prices from API Request p.3 (NEW)
23.Project #5: Web Scraping (UPDATED) (01 hour 00 minutes)LinkedIn Scraping Project: Overview & Requirements (UPDATED)
LinkedIn Logging in (UPDATED)
Finding LinkedIn Profiles: Part 1 (UPDATED)
Finding LinkedIn Profiles: Part 2 (UPDATED)
Scraping Data Points from LinkedIn Profiles: Part 1 (UPDATED)
Scraping Data Points from LinkedIn Profiles: Part 2 (UPDATED)
Connecting to LinkedIn Profiles (UPDATED)
24.Solved Web Scraping Exercises (02 minutes)Yield Data Items from 2 Functions
How to Order Exported Data
Xpath contains() and starts-with() functions
25.Bonus: Data Extraction with APIs (17 seconds)Data Extraction with APIs (Free Tutorial)
26.Bonus: Web Scraping with Beautiful Soup, Requests & Selenium Course (01 minutes)Coupon for Web Scraping with Beautiful Soup, Requests & Selenium & Other Courses

Resources Required

  • This Scrapy course presupposes that you are already familiar with the fundamentals of developing straightforward Python programmes and with the language’s essential capabilities (data structures, file handling, functions, classes, modules, common libraries, etc)
  • Python 2.7 or Python 3.3
  • The best operating system is any one (Linux, Mac, Windows)
  • Positive attitude, openness to learning new things, and willingness to ask questions (if any) during the course’s Q&A session
  • Please read the course description and watch the preview lectures Before enrolling in the course if you have any questions about what Scrapy is or why you should use it.

Featured Review

Umar Ibrahim (4/5) : the course is excellent for me and it has provided me the best of what I need to be an expert in data scraping with python


  • Andre Da Rocha (5/5) : This course is by bar the best on teaching how to use scrapy and the basics of web scraping.
  • Qingling Ni (4/5) : This course is probably the best to learn how to use Scrapy.
  • ATM Sayfuddin (5/5) : The instructor responds promptly and the best thing about this course is that I can progress through my project as I learn during the course.
  • Anjaneya Prasad Nidubrolu (5/5) : I felt the course is awesome and tutor is at his best.


  • Yang Chenjie (1/5) : If I have no prior knowledge, I can hardly understand what the instructor is talking about.
  • Anonymized User (1/5) : None of the commands work in windows, was not aware that a prerequisite for this course was to have a Ubuntu Linux computer.
  • Abhishek Tripathi (1/5) : While the course has alot of content, it doesn’t explain what certain things mean and how they work. Q&A is Slow aswell, its been 2 days since i asked a question and i did not get a answer yet, had to buy another course from someone else just to understand how to solve my problem

About the Author

The instructor of this course is GoTrained Academy who is a eLearning Professionals. With 3.7 Instructor Rating and 6,365 Reviews on Udemy, GoTrained Academy offers 9 Courses and has taught 74,624 Students so far.

  • GoTrained is an e-learning academy that focuses on technology and management while producing helpful information in several languages
  • GoTrained Academy use a unique method for choosing the content we offer, they primarily concentrate on abilities that are regularly required by clients and jobs even though there aren’t many videos that teach them
  • GoTrained Academy also make an effort to create video series that cover both the fundamentals and more advanced topics

Comparison Table

ParametersScrapy: Powerful Web Scraping & Crawling with PythonAdvanced Web Scraping with Python using Scrapy & SplashModern Web Scraping with Python using Scrapy Splash Selenium
OffersINR 499 (INR 2,599) 78% offINR 455 (INR 3,499) 87% offINR 455 (INR 3,499) 87% off
Duration10.5 hours5.5 hours9 hours
Student Enrollments16,1336,16022,790
InstructorsGoTrained AcademyAhmed RafikAhmed Rafik
Register HereApply Now!Apply Now!Apply Now!

Leave feedback about this

  • Rating