ejemplo de código del tutorial de scrapy python

Te recomendamos que pruebes esta solución en un ambiente controlado antes de pasarlo a producción, saludos.

Ejemplo 1: rastreador web de Python

import scrapy

classBlogSpider(scrapy.Spider):
    name ='blogspider'
    start_urls =['https://blog.scrapinghub.com']defparse(self, response):for title in response.css('.post-header>h2'):yield'title': title.css('a ::text').get()for next_page in response.css('a.next-posts-link'):yield response.follow(next_page, self.parse)

Ejemplo 2: proyecto de creación de scrapy

scrapy startproject projectname

Ejemplo 3: tutorial de scrapy

# -*- coding: utf-8 -*-import scrapy


classAliexpressTabletsSpider(scrapy.Spider):
    name ='aliexpress_tablets'
    allowed_domains =['aliexpress.com']
    start_urls =['https://www.aliexpress.com/category/200216607/tablets.html','https://www.aliexpress.com/category/200216607/tablets/2.html?site=glo&g=y&tag=']defparse(self, response):print("procesing:"+response.url)#Extract data using css selectors
        product_name=response.css('.product::text').extract()
        price_range=response.css('.value::text').extract()#Extract data using xpath
        orders=response.xpath("//em[@title='Total Orders']/text()").extract()
        company_name=response.xpath("//a[@class='store $p4pLog']/text()").extract()

        row_data=zip(product_name,price_range,orders,company_name)#Making extracted data row wisefor item in row_data:#create a dictionary to store the scraped info
            scraped_info =#key:value'page':response.url,'product_name': item[0],#item[0] means product in the list and so on, index tells what value to assign'price_range': item[1],'orders': item[2],'company_name': item[3],#yield or give the scraped info to scrapyyield scraped_info

Si te animas, puedes dejar un tutorial acerca de qué le añadirías a este post.

---

---

Utiliza Nuestro Buscador