The ‘Modern Web Scraping with Python using Scrapy Splash Selenium’ course teaches the fundamentals of Web Scraping. In this course, you will learn how to build a complete Spider from A to Z.

The course also teaches how to store the extracted data in MongoDb & SQLite3 and how to scrape JavaScript websites using Splash & Selenium.  Along with that, you will learn how to host spiders in Heroku as well as Splash. The course is usually available for INR 2,799 on Udemy but you can click on the link and get the ‘Modern Web Scraping with Python using Scrapy Splash Selenium’ for INR 449.

Who all can opt for this course?

  • Anybody who desires to scrape information from any website
  • Anyone desires to learn Scrapy
  • Anybody wishes to automate the process of copying content from websites
  • Everyone who wants to learn how to use Scrapy-Splash and Selenium to scrape Javascript websites

Course Highlights

Key HighlightsDetails
Registration LinkApply Now!
PriceINR 449 (INR 2,79980% off
Duration8.5 Hours
Student Enrollment22,790 students
InstructorAhmed Rafik
Topics CoveredScrapy Fundamentals, Xpath, CSS Selector, JSON, Scrapy Splash & Selenium
Course LevelIntermediate
Total Student Reviews3,583

Learning Outcomes

  • Recognize the basics of web scraping
  • Website scraping with Scrapy
  • Comprehend CSS selectors and Xpath
  • Construct a Spider from beginning to end
  • Put the extracted data in SQLite3 and MongoDb
  • Splash and Selenium can be used to scrape JavaScript websites
  • Construct a CrawlSpider
  • Recognize the behaviour of crawling
  • Create a unique Middleware
  • Recommended practises for web scraping
  • When scraping websites, avoid getting blacklisted
  • Circumvent cloudflare
  • Snoop on APIs
  • Scrape websites with unlimited scroll
  • Using Cookies
  • Both locally and on the cloud, deploy spiders
  • Run spiders on a regular basis
  • Create datasets
  • Use Scrapy to log into websites
  • Use Scrapy to download files and photos

Course Content

S.No.Module (Duration)Topics
1.Introduction (17 minutes)Intro to Web Scraping & Scrapy
Setting up Scrapy the Development Environment (Updated)
Add VSCODE to path (Mac users)
Udemy 101 (Please don’t skip*)
Asking questions
2.Scrapy Fundamentals (30 minutes)Scrapy fundamentals PART 1
Scrapy fundamentals PART 2
Scrapy fundamentals PART 3
Scrapy fundamentals PART 4
Scrapy fundamentals PART 5
3.XPath expressions & CSS Selectors (36 minutes)Downloadable files
XPath & CSS Selectors
CSS Selectors fundamentals
CSS selectors in theory
XPath fundamentals
Navigating using XPath(Going UP)
Navigating using XPath(Going DOWN)
XPath in theory
4.Project 1 Spiders from A to Z (21 minutes)Worldometers PART 1
Worldometers PART 2
Worldometers PART 3
Worldometers PART 4
Project source code
5.Building Datasets (04 minutes)Bulding datesets
6.Project 2 Dealing with Multiple pages (23 minutes)Website URL (Please do not skip)
Setting up the project
Setting up the project – Code update –
Building the spider
Dealing with pagination
Spoofing request headers
TinyDeal project source code
Exercise 2
7.Debugging spiders (15 minutes)What is debugging?
Debugging spiders PART 1
Debugging spiders PART 2
8.Let’s take a break ! (04 minutes)The “whys” & “whens” of web scraping
Web scraping challenges
9.Project 3 Build Crawlers using Scrapy (21 minutes)Website URL update
Crawl spider structure
The Rule object
Following links in pagination
Spoofing request headers
Project source code
10.Splash crash course (30 minutes)What dilemma splash came to solve
Setting up Splash (Windows Pro/Entreprise edition & Mac Os)
Setting up Splash(Windows Home Edition)
Setting up Splash (Linux)
Introduction to Splash
Working with elements
Spoofing request headers
11.Project 4 Scraping JavaScript websites using Splash (16 minutes)Website URL update
Splash incognito mode
Using Splash with Scrapy
Project source code
12.Project 5 Scraping JavaScript websites using Selenium (54 minutes)Selenium basics
ElementNotInteractable Exception
Selenium with Scrapy
Selenium Middleware PART 1 (NEW)
Selenium Middleware PART 2 (NEW)
Project source code
13.Working with Pipelines (22 minutes)Pipelines
Storing data in MongoDB
Storing data in SQLite3
Project source code
14.Scraping APIs (NEW) (23 minutes)Scraping APIs PART 1
Scraping APIs PART 2
Scraping APIs PART 3
Scraping APIs PART 4
Scraping APIs PART 5
Project source code
15.Log in to websites (NEW) (19 minutes)Log in to websites PART 1
Log in to websites PART 2
Log in to websites PART 3 (JavaScript required)
Project source code
16.Project 6 Bypass Cloudflare (13 minutes)Website URL update
Bypass Cloudflare PART 1
Bypass Cloudflare PART 2
Project source code
17.APPENDIX (OLDER SCRAPY 1.5 CONTENT) (02 hours 57 minutes)*IMPORTANT*
Avoid getting banned PART 1
Avoid getting banned PART 2
Avoid getting banned PART 3
Scraping APIs PART 1
Scraping APIs PART 2
Scraping APIs PART 3
Scraping APIs PART 4
Hidden XHR
Scraping APIs PART 5
Scraping APIs PART 6
Spider Arguments
Scraping APIs PART 7
Another way to scrape Airbnb restaurant detail page
Deploying spiders PART 1
Deploying spiders PART 2
Deploying spiders PART 3
Deploying spiders PART 4
Execute spiders periodically
Deploy Splash to Heroku
Project source code
Project source code
Challenge for those who are adventurous
Login to websites using FormRequest
XML Http Post Requests
Project source code
Code UPDATE XHR repeated data (Assignment)
Media Pipelines
The Images Pipeline
Extending The Images Pipeline (Store images with custom names)
Files Pipeline (Article)
Challenge (Files Pipeline)
Project source code
Using Crawlera with Scrapy
Using Crawlera with Splash
Using Heroku as a Proxy (FREE)
Using FREE Proxies with the CrawlSpider
Project source code
18.BONUS (18 seconds)Files Pipeline
Bonus Lecture

Resources Required

  • Python’s fundamentals
  • Internet connected-computer

Featured Review

Meeran Muhaiyaddin Muhammed (5/5) : One of the best ever Web scraping course I ever took on Udemy. It helped me alot in developing my web scraping skills in my work environment.


  • Shlomit Dror (5/5) : I am deeply impressed by the quality of this course; the videos and teacher’s explanations.
  • Misha Student (5/5) : Really excellent course that walks you through setting up web scraping spiders for your projects.
  • Okodi Ataime Benson (5/5) : This is the best web scraping course I have ever taken.
  • Rémi Caland (5/5) : He takes time to explain a lot of part of his work, and this course is perfectly made.


  • Priyadarshan Gupta (2/5) : Felt like, we were supposed to just follow instructions blindly without questioning.
  • Unnamed Student (1/5) : Already left my comments about this horrible course in the Q&A section already.
  • Utku Çaml?da? (1/5) : Well due to lack of update of splash this course is now basicly useless.
  • Unnamed Student (1/5) : I’m not one to disparage instructors on here because Udemy has been a value-added for me for the past one and a half years but dude, u really do not know how to teach.

About the Author

The instructor of this course is Ahmed Rafik who is a Developer and Online Instructor. With 4.4 Instructor Rating and 4,380 Reviews on Udemy, Instructor offers 3 Courses and has taught 31,518 Students so far.

  • Ahmed Rafik, teach online courses through Udemy as a self-taught developer
  • With the use of various programmes like Scrapy, Splash, and Selenium, he has assisted hundreds of people in learning how to perform web scraping using Python
  • Ahmed Rafik think learning to code should be simple for everyone, but need to pick a teacher that has the proper skills
  • With his courses, he provide you the practical knowledge that you need to launch a successful web scraping business right away
  • The fact that Ahmed Rafik always keep his courses up to date and do his best to avoid dull theoretical explanations wherever feasible is what has allowed him to have the highest rated and best-selling web scraping courses on Udemy
  • Ahmed Rafik look forward to having you enrolled in one of his courses; he will make sure to support you at every turn and respond to your inquiries

Comparison Table

ParametersModern Web Scraping with Python using Scrapy Splash SeleniumScrapy: Powerful Web Scraping & Crawling with PythonAdvanced Web Scraping with Python using Scrapy & Splash
OffersINR 449 (INR 2,799) 80% offINR 455 (INR 3,499) 87% offINR 455 (INR 3,499) 87% off
Duration8.5 hours11 hours5.5 hours
Rating4.3/54.3 /54.8 /5
Student Enrollments22,79016,1336,160
InstructorsAhmed RafikGoTrained AcademyAhmed Rafik
Register HereApply Now!Apply Now!Apply Now!

Leave feedback about this

  • Rating