In the last part of my Web Scraping series, we will refactor the Spider and fix the issues we have encountered in the previous parts. Also, we will deploy and run our Scrapy Spider on Jenkins.
This article is the third part of my web scraping series. I focuse on creating a Scrapy Spider to scrape Apple's website for the latest iPhone models and their prices. I will explain how to create a new Scrapy project, and generate a Spider. The article also covers fetching the pages for different iPhone models from the US, Swedish, and Hungarian Apple sites.
The 2nd part of my web scraping tutorial covers Scrapy Shell, which can be used to interactively test and refine scraping logic. We will scrape Apple's website for the latest iPhone models and their prices.
