2024-09-26 18:17:52 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: catalog_discovery)
2024-09-26 18:17:52 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.9 (main, Aug 13 2024, 01:19:58) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-6.4.10-dirty-x86_64-with-glibc2.36
2024-09-26 18:17:52 [scrapy.addons] INFO: Enabled addons:
[]
2024-09-26 18:17:52 [scrapy.extensions.telnet] INFO: Telnet Password: 10e102d56b4db8fe
2024-09-26 18:17:52 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats',
 'spidermon.contrib.scrapy.extensions.Spidermon']
2024-09-26 18:17:52 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'catalog_discovery',
 'CONCURRENT_ITEMS': 1000,
 'CONCURRENT_REQUESTS': 32,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/var/lib/scrapyd/logs/catalog_discovery/lowes/a08708fc7c3311efa9cf4200a9fe0102.log',
 'LOG_LEVEL': 'INFO',
 'NEWSPIDER_MODULE': 'catalog_discovery.spiders',
 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_poet.ScrapyPoetRequestFingerprinter',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'RETRY_TIMES': 5,
 'SPIDER_MODULES': ['catalog_discovery.spiders'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
 'USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:125.0) '
               'Gecko/20100101 Firefox/125.0'}
2024-09-26 18:17:53 [scrapy_poet.injection] INFO: Loading providers:
 [<class 'scrapy_poet.page_input_providers.HttpRequestProvider'>,
 <class 'scrapy_poet.page_input_providers.HttpResponseProvider'>,
 <class 'scrapy_poet.page_input_providers.HttpClientProvider'>,
 <class 'scrapy_poet.page_input_providers.PageParamsProvider'>,
 <class 'scrapy_poet.page_input_providers.RequestUrlProvider'>,
 <class 'scrapy_poet.page_input_providers.ResponseUrlProvider'>,
 <class 'scrapy_poet.page_input_providers.StatsProvider'>]
2024-09-26 18:17:53 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy_poet.InjectionMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy_poet.DownloaderStatsMiddleware']
2024-09-26 18:17:53 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy_poet.RetryMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2024-09-26 18:17:53 [scrapy.middleware] INFO: Enabled item pipelines:
['scraping_utils.pipelines.DuplicatesFilterPipeline',
 'scraping_utils.pipelines.AttachSupplierPipeline',
 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline']
2024-09-26 18:17:53 [scrapy.core.engine] INFO: Spider opened
2024-09-26 18:17:53 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2024-09-26 18:17:53 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2024-09-26 18:18:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:1
2024-09-26 18:18:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:2
2024-09-26 18:18:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:3
2024-09-26 18:18:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:4
2024-09-26 18:18:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:5
2024-09-26 18:18:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:6
2024-09-26 18:18:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:7
2024-09-26 18:18:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:8
2024-09-26 18:18:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:9
2024-09-26 18:18:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:10
2024-09-26 18:18:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:11
2024-09-26 18:18:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:12
2024-09-26 18:18:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:13
2024-09-26 18:18:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:14
2024-09-26 18:18:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:15
2024-09-26 18:18:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:16
2024-09-26 18:18:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:17
2024-09-26 18:18:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:18
2024-09-26 18:18:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:19
2024-09-26 18:18:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:20
2024-09-26 18:18:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:21
2024-09-26 18:18:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:22
2024-09-26 18:18:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:23
2024-09-26 18:18:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:24
2024-09-26 18:18:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:25
2024-09-26 18:18:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:26
2024-09-26 18:18:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:27
2024-09-26 18:18:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:28
2024-09-26 18:18:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:29
2024-09-26 18:18:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:30
2024-09-26 18:18:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:31
2024-09-26 18:18:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:32
2024-09-26 18:18:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:33
2024-09-26 18:18:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:34
2024-09-26 18:18:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:35
2024-09-26 18:18:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:36
2024-09-26 18:18:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:37
2024-09-26 18:18:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:38
2024-09-26 18:18:53 [scrapy.extensions.logstats] INFO: Crawled 33 pages (at 33 pages/min), scraped 97960 items (at 97960 items/min)
2024-09-26 18:18:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:39
2024-09-26 18:18:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:40
2024-09-26 18:18:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:41
2024-09-26 18:18:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:42
2024-09-26 18:18:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:43
2024-09-26 18:18:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:44
2024-09-26 18:19:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:45
2024-09-26 18:19:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:46
2024-09-26 18:19:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:47
2024-09-26 18:19:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:48
2024-09-26 18:19:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:49
2024-09-26 18:19:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:50
2024-09-26 18:19:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:51
2024-09-26 18:19:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:52
2024-09-26 18:19:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:53
2024-09-26 18:19:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:54
2024-09-26 18:19:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:55
2024-09-26 18:19:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:56
2024-09-26 18:19:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:57
2024-09-26 18:19:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:58
2024-09-26 18:19:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:59
2024-09-26 18:19:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:60
2024-09-26 18:19:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:61
2024-09-26 18:19:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:62
2024-09-26 18:19:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:63
2024-09-26 18:19:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:64
2024-09-26 18:19:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:65
2024-09-26 18:19:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:66
2024-09-26 18:19:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:67
2024-09-26 18:19:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:68
2024-09-26 18:19:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:69
2024-09-26 18:19:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:70
2024-09-26 18:19:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:71
2024-09-26 18:19:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:72
2024-09-26 18:19:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:73
2024-09-26 18:19:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:74
2024-09-26 18:19:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:75
2024-09-26 18:19:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:76
2024-09-26 18:19:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:77
2024-09-26 18:19:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:78
2024-09-26 18:19:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:79
2024-09-26 18:19:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:80
2024-09-26 18:19:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:81
2024-09-26 18:19:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:82
2024-09-26 18:19:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:83
2024-09-26 18:19:53 [scrapy.extensions.logstats] INFO: Crawled 33 pages (at 0 pages/min), scraped 208723 items (at 110763 items/min)
2024-09-26 18:19:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:84
2024-09-26 18:19:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:85
2024-09-26 18:19:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:86
2024-09-26 18:19:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:87
2024-09-26 18:19:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:88
2024-09-26 18:20:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:89
2024-09-26 18:20:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:90
2024-09-26 18:20:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:91
2024-09-26 18:20:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:92
2024-09-26 18:20:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:93
2024-09-26 18:20:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:94
2024-09-26 18:20:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:95
2024-09-26 18:20:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:96
2024-09-26 18:20:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:97
2024-09-26 18:20:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:98
2024-09-26 18:20:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:99
2024-09-26 18:20:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:100
2024-09-26 18:20:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:101
2024-09-26 18:20:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:102
2024-09-26 18:20:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:103
2024-09-26 18:20:27 [scrapy.core.scraper] WARNING: Dropped: Validation failed!
{'url': 'https://www.lowes.com/pd/origin 21-28-in-L-x-28-in-W-Round-Gold-Framed-Wall-Mirror/1000421755', 'sku': None, 'name': None, 'supplier': 'LOWES'}
2024-09-26 18:20:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:104
2024-09-26 18:20:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:105
2024-09-26 18:20:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:106
2024-09-26 18:20:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:107
2024-09-26 18:20:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:108
2024-09-26 18:20:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:109
2024-09-26 18:20:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:110
2024-09-26 18:20:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:111
2024-09-26 18:20:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:112
2024-09-26 18:20:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:113
2024-09-26 18:20:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:114
2024-09-26 18:20:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:115
2024-09-26 18:20:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:116
2024-09-26 18:20:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:117
2024-09-26 18:20:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:118
2024-09-26 18:20:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:119
2024-09-26 18:20:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:120
2024-09-26 18:20:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:121
2024-09-26 18:20:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:122
2024-09-26 18:20:53 [scrapy.extensions.logstats] INFO: Crawled 65 pages (at 32 pages/min), scraped 306562 items (at 97839 items/min)
2024-09-26 18:20:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:123
2024-09-26 18:20:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:124
2024-09-26 18:20:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:125
2024-09-26 18:20:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:126
2024-09-26 18:20:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:127
2024-09-26 18:21:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:128
2024-09-26 18:21:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:129
2024-09-26 18:21:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:130
2024-09-26 18:21:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:131
2024-09-26 18:21:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:132
2024-09-26 18:21:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:133
2024-09-26 18:21:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:134
2024-09-26 18:21:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:135
2024-09-26 18:21:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:136
2024-09-26 18:21:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:137
2024-09-26 18:21:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:138
2024-09-26 18:21:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:139
2024-09-26 18:21:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:140
2024-09-26 18:21:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:141
2024-09-26 18:21:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:142
2024-09-26 18:21:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:143
2024-09-26 18:21:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:144
2024-09-26 18:21:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:145
2024-09-26 18:21:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:146
2024-09-26 18:21:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:147
2024-09-26 18:21:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:148
2024-09-26 18:21:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:149
2024-09-26 18:21:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:150
2024-09-26 18:21:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:152
2024-09-26 18:21:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:151
2024-09-26 18:21:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:153
2024-09-26 18:21:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:154
2024-09-26 18:21:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:155
2024-09-26 18:21:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:156
2024-09-26 18:21:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:157
2024-09-26 18:21:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:158
2024-09-26 18:21:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:159
2024-09-26 18:21:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:160
2024-09-26 18:21:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:161
2024-09-26 18:21:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:162
2024-09-26 18:21:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:163
2024-09-26 18:21:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:164
2024-09-26 18:21:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:165
2024-09-26 18:21:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:166
2024-09-26 18:21:53 [scrapy.extensions.logstats] INFO: Crawled 65 pages (at 0 pages/min), scraped 416109 items (at 109547 items/min)
2024-09-26 18:21:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:167
2024-09-26 18:21:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:168
2024-09-26 18:21:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:169
2024-09-26 18:21:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:170
2024-09-26 18:22:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:171
2024-09-26 18:22:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:172
2024-09-26 18:22:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:173
2024-09-26 18:22:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:174
2024-09-26 18:22:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:175
2024-09-26 18:22:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:176
2024-09-26 18:22:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:177
2024-09-26 18:22:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:178
2024-09-26 18:22:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:179
2024-09-26 18:22:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:180
2024-09-26 18:22:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:181
2024-09-26 18:22:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:182
2024-09-26 18:22:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:183
2024-09-26 18:22:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:184
2024-09-26 18:22:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:185
2024-09-26 18:22:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:186
2024-09-26 18:22:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:187
2024-09-26 18:22:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:188
2024-09-26 18:22:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:189
2024-09-26 18:22:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:190
2024-09-26 18:22:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:191
2024-09-26 18:22:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:192
2024-09-26 18:22:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:193
2024-09-26 18:22:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:194
2024-09-26 18:22:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:195
2024-09-26 18:22:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:196
2024-09-26 18:22:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:197
2024-09-26 18:22:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:198
2024-09-26 18:22:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:199
2024-09-26 18:22:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:200
2024-09-26 18:22:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:201
2024-09-26 18:22:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:202
2024-09-26 18:22:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:203
2024-09-26 18:22:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:204
2024-09-26 18:22:53 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 32 pages/min), scraped 512160 items (at 96051 items/min)
2024-09-26 18:22:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:205
2024-09-26 18:22:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:206
2024-09-26 18:22:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:207
2024-09-26 18:22:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:208
2024-09-26 18:22:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:209
2024-09-26 18:23:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:210
2024-09-26 18:23:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:211
2024-09-26 18:23:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:212
2024-09-26 18:23:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:213
2024-09-26 18:23:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:214
2024-09-26 18:23:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:215
2024-09-26 18:23:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:216
2024-09-26 18:23:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:217
2024-09-26 18:23:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:218
2024-09-26 18:23:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:219
2024-09-26 18:23:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:220
2024-09-26 18:23:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:221
2024-09-26 18:23:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:222
2024-09-26 18:23:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:223
2024-09-26 18:23:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:224
2024-09-26 18:23:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:225
2024-09-26 18:23:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:226
2024-09-26 18:23:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:227
2024-09-26 18:23:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:228
2024-09-26 18:23:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:229
2024-09-26 18:23:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:230
2024-09-26 18:23:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:231
2024-09-26 18:23:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:232
2024-09-26 18:23:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:233
2024-09-26 18:23:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:234
2024-09-26 18:23:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:235
2024-09-26 18:23:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:236
2024-09-26 18:23:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:237
2024-09-26 18:23:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:238
2024-09-26 18:23:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:239
2024-09-26 18:23:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:240
2024-09-26 18:23:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:241
2024-09-26 18:23:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:242
2024-09-26 18:23:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:243
2024-09-26 18:23:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:244
2024-09-26 18:23:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:245
2024-09-26 18:23:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:246
2024-09-26 18:23:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:247
2024-09-26 18:23:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:248
2024-09-26 18:23:53 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 0 pages/min), scraped 622912 items (at 110752 items/min)
2024-09-26 18:23:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:249
2024-09-26 18:23:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:250
2024-09-26 18:23:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:251
2024-09-26 18:23:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:252
2024-09-26 18:23:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:253
2024-09-26 18:24:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:254
2024-09-26 18:24:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:255
2024-09-26 18:24:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:256
2024-09-26 18:24:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:257
2024-09-26 18:24:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:258
2024-09-26 18:24:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:259
2024-09-26 18:24:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:260
2024-09-26 18:24:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:261
2024-09-26 18:24:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:262
2024-09-26 18:24:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:263
2024-09-26 18:24:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:264
2024-09-26 18:24:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:265
2024-09-26 18:24:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:266
2024-09-26 18:24:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:267
2024-09-26 18:24:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:268
2024-09-26 18:24:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:269
2024-09-26 18:24:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:270
2024-09-26 18:24:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:271
2024-09-26 18:24:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:272
2024-09-26 18:24:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:273
2024-09-26 18:24:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:274
2024-09-26 18:24:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:275
2024-09-26 18:24:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:276
2024-09-26 18:24:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:277
2024-09-26 18:24:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:278
2024-09-26 18:24:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:279
2024-09-26 18:24:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:280
2024-09-26 18:24:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:281
2024-09-26 18:24:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:282
2024-09-26 18:24:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:283
2024-09-26 18:24:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:284
2024-09-26 18:24:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:285
2024-09-26 18:24:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:286
2024-09-26 18:24:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:287
2024-09-26 18:24:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:288
2024-09-26 18:24:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:289
2024-09-26 18:24:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:290
2024-09-26 18:24:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:291
2024-09-26 18:24:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:292
2024-09-26 18:24:53 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 0 pages/min), scraped 732896 items (at 109984 items/min)
2024-09-26 18:24:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:293
2024-09-26 18:24:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:294
2024-09-26 18:24:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:295
2024-09-26 18:24:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:296
2024-09-26 18:24:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:297
2024-09-26 18:25:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:298
2024-09-26 18:25:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:299
2024-09-26 18:25:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:300
2024-09-26 18:25:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:301
2024-09-26 18:25:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:302
2024-09-26 18:25:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:303
2024-09-26 18:25:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:304
2024-09-26 18:25:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:305
2024-09-26 18:25:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:306
2024-09-26 18:25:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:307
2024-09-26 18:25:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:308
2024-09-26 18:25:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:309
2024-09-26 18:25:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:310
2024-09-26 18:25:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:311
2024-09-26 18:25:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:312
2024-09-26 18:25:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:313
2024-09-26 18:25:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:314
2024-09-26 18:25:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:315
2024-09-26 18:25:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:316
2024-09-26 18:25:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:317
2024-09-26 18:25:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:318
2024-09-26 18:25:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:319
2024-09-26 18:25:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:320
2024-09-26 18:25:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:321
2024-09-26 18:25:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:322
2024-09-26 18:25:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:323
2024-09-26 18:25:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:324
2024-09-26 18:25:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:325
2024-09-26 18:25:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:326
2024-09-26 18:25:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:327
2024-09-26 18:25:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:328
2024-09-26 18:25:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:329
2024-09-26 18:25:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:330
2024-09-26 18:25:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:331
2024-09-26 18:25:53 [scrapy.extensions.logstats] INFO: Crawled 129 pages (at 32 pages/min), scraped 828192 items (at 95296 items/min)
2024-09-26 18:25:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:332
2024-09-26 18:25:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:333
2024-09-26 18:25:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:334
2024-09-26 18:25:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:335
2024-09-26 18:26:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:336
2024-09-26 18:26:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:337
2024-09-26 18:26:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:338
2024-09-26 18:26:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:339
2024-09-26 18:26:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:340
2024-09-26 18:26:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:341
2024-09-26 18:26:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:342
2024-09-26 18:26:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:343
2024-09-26 18:26:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:344
2024-09-26 18:26:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:345
2024-09-26 18:26:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:346
2024-09-26 18:26:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:347
2024-09-26 18:26:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:348
2024-09-26 18:26:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:349
2024-09-26 18:26:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:350
2024-09-26 18:26:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:351
2024-09-26 18:26:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:352
2024-09-26 18:26:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:353
2024-09-26 18:26:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:354
2024-09-26 18:26:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:355
2024-09-26 18:26:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:356
2024-09-26 18:26:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:357
2024-09-26 18:26:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:358
2024-09-26 18:26:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:359
2024-09-26 18:26:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:360
2024-09-26 18:26:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:361
2024-09-26 18:26:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:362
2024-09-26 18:26:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:363
2024-09-26 18:26:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:364
2024-09-26 18:26:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:365
2024-09-26 18:26:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:366
2024-09-26 18:26:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:367
2024-09-26 18:26:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:368
2024-09-26 18:26:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:369
2024-09-26 18:26:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:370
2024-09-26 18:26:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:371
2024-09-26 18:26:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:372
2024-09-26 18:26:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:373
2024-09-26 18:26:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:374
2024-09-26 18:26:53 [scrapy.extensions.logstats] INFO: Crawled 129 pages (at 0 pages/min), scraped 938112 items (at 109920 items/min)
2024-09-26 18:26:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:375
2024-09-26 18:26:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:376
2024-09-26 18:26:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:377
2024-09-26 18:26:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:378
2024-09-26 18:26:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:379
2024-09-26 18:26:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:380
2024-09-26 18:27:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:381
2024-09-26 18:27:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:382
2024-09-26 18:27:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:383
2024-09-26 18:27:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:384
2024-09-26 18:27:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:385
2024-09-26 18:27:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:386
2024-09-26 18:27:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:387
2024-09-26 18:27:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:388
2024-09-26 18:27:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:389
2024-09-26 18:27:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:390
2024-09-26 18:27:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:391
2024-09-26 18:27:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:392
2024-09-26 18:27:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:393
2024-09-26 18:27:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:394
2024-09-26 18:27:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:395
2024-09-26 18:27:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:396
2024-09-26 18:27:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:397
2024-09-26 18:27:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:398
2024-09-26 18:27:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:399
2024-09-26 18:27:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:400
2024-09-26 18:27:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:401
2024-09-26 18:27:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:402
2024-09-26 18:27:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:403
2024-09-26 18:27:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:404
2024-09-26 18:27:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:405
2024-09-26 18:27:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:406
2024-09-26 18:27:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:407
2024-09-26 18:27:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:408
2024-09-26 18:27:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:409
2024-09-26 18:27:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:410
2024-09-26 18:27:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:411
2024-09-26 18:27:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:412
2024-09-26 18:27:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:413
2024-09-26 18:27:53 [scrapy.extensions.logstats] INFO: Crawled 161 pages (at 32 pages/min), scraped 1035247 items (at 97135 items/min)
2024-09-26 18:27:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:414
2024-09-26 18:27:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:415
2024-09-26 18:27:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:416
2024-09-26 18:27:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:417
2024-09-26 18:27:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:418
2024-09-26 18:28:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:419
2024-09-26 18:28:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:420
2024-09-26 18:28:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:421
2024-09-26 18:28:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:422
2024-09-26 18:28:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:423
2024-09-26 18:28:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:424
2024-09-26 18:28:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:425
2024-09-26 18:28:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:426
2024-09-26 18:28:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:427
2024-09-26 18:28:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:428
2024-09-26 18:28:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:429
2024-09-26 18:28:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:430
2024-09-26 18:28:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:431
2024-09-26 18:28:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:432
2024-09-26 18:28:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:433
2024-09-26 18:28:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:434
2024-09-26 18:28:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:435
2024-09-26 18:28:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:436
2024-09-26 18:28:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:437
2024-09-26 18:28:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:438
2024-09-26 18:28:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:439
2024-09-26 18:28:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:440
2024-09-26 18:28:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:441
2024-09-26 18:28:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:442
2024-09-26 18:28:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:443
2024-09-26 18:28:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:444
2024-09-26 18:28:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:445
2024-09-26 18:28:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:446
2024-09-26 18:28:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:447
2024-09-26 18:28:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:448
2024-09-26 18:28:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:449
2024-09-26 18:28:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:450
2024-09-26 18:28:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:451
2024-09-26 18:28:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:452
2024-09-26 18:28:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:453
2024-09-26 18:28:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:454
2024-09-26 18:28:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:455
2024-09-26 18:28:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:456
2024-09-26 18:28:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:457
2024-09-26 18:28:53 [scrapy.extensions.logstats] INFO: Crawled 161 pages (at 0 pages/min), scraped 1144559 items (at 109312 items/min)
2024-09-26 18:28:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:458
2024-09-26 18:28:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:459
2024-09-26 18:28:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:460
2024-09-26 18:28:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:461
2024-09-26 18:28:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:462
2024-09-26 18:29:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:463
2024-09-26 18:29:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:464
2024-09-26 18:29:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:465
2024-09-26 18:29:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:466
2024-09-26 18:29:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:467
2024-09-26 18:29:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:468
2024-09-26 18:29:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:469
2024-09-26 18:29:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:470
2024-09-26 18:29:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:471
2024-09-26 18:29:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:472
2024-09-26 18:29:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:473
2024-09-26 18:29:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:474
2024-09-26 18:29:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:475
2024-09-26 18:29:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:476
2024-09-26 18:29:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:477
2024-09-26 18:29:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:478
2024-09-26 18:29:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:479
2024-09-26 18:29:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:480
2024-09-26 18:29:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:481
2024-09-26 18:29:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:482
2024-09-26 18:29:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:483
2024-09-26 18:29:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:484
2024-09-26 18:29:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:485
2024-09-26 18:29:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:486
2024-09-26 18:29:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:487
2024-09-26 18:29:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:488
2024-09-26 18:29:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:489
2024-09-26 18:29:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:490
2024-09-26 18:29:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:491
2024-09-26 18:29:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:492
2024-09-26 18:29:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:493
2024-09-26 18:29:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:494
2024-09-26 18:29:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:495
2024-09-26 18:29:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:496
2024-09-26 18:29:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:497
2024-09-26 18:29:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:498
2024-09-26 18:29:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:499
2024-09-26 18:29:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:500
2024-09-26 18:29:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:501
2024-09-26 18:29:53 [scrapy.extensions.logstats] INFO: Crawled 161 pages (at 0 pages/min), scraped 1254100 items (at 109541 items/min)
2024-09-26 18:29:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:502
2024-09-26 18:30:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:503
2024-09-26 18:30:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:504
2024-09-26 18:30:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:505
2024-09-26 18:30:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:506
2024-09-26 18:30:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:507
2024-09-26 18:30:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:508
2024-09-26 18:30:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:509
2024-09-26 18:30:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:510
2024-09-26 18:30:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:511
2024-09-26 18:30:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:512
2024-09-26 18:30:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:513
2024-09-26 18:30:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:514
2024-09-26 18:30:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:515
2024-09-26 18:30:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:516
2024-09-26 18:30:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:517
2024-09-26 18:30:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:518
2024-09-26 18:30:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:519
2024-09-26 18:30:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:520
2024-09-26 18:30:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:521
2024-09-26 18:30:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:522
2024-09-26 18:30:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:523
2024-09-26 18:30:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:524
2024-09-26 18:30:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:525
2024-09-26 18:30:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:526
2024-09-26 18:30:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:527
2024-09-26 18:30:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:528
2024-09-26 18:30:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:529
2024-09-26 18:30:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:530
2024-09-26 18:30:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:531
2024-09-26 18:30:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:532
2024-09-26 18:30:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:533
2024-09-26 18:30:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:534
2024-09-26 18:30:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:535
2024-09-26 18:30:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:536
2024-09-26 18:30:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:537
2024-09-26 18:30:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:538
2024-09-26 18:30:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:539
2024-09-26 18:30:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:540
2024-09-26 18:30:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:541
2024-09-26 18:30:53 [scrapy.extensions.logstats] INFO: Crawled 185 pages (at 24 pages/min), scraped 1354560 items (at 100460 items/min)
2024-09-26 18:30:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:542
2024-09-26 18:30:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:543
2024-09-26 18:30:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:544
2024-09-26 18:30:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:545
2024-09-26 18:30:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:546
2024-09-26 18:31:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:547
2024-09-26 18:31:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:548
2024-09-26 18:31:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:549
2024-09-26 18:31:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:550
2024-09-26 18:31:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:551
2024-09-26 18:31:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:552
2024-09-26 18:31:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:553
2024-09-26 18:31:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:554
2024-09-26 18:31:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:555
2024-09-26 18:31:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:556
2024-09-26 18:31:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:557
2024-09-26 18:31:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:558
2024-09-26 18:31:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:559
2024-09-26 18:31:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:560
2024-09-26 18:31:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:561
2024-09-26 18:31:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:562
2024-09-26 18:31:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:563
2024-09-26 18:31:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:564
2024-09-26 18:31:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:565
2024-09-26 18:31:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:566
2024-09-26 18:31:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:567
2024-09-26 18:31:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:568
2024-09-26 18:31:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:569
2024-09-26 18:31:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:570
2024-09-26 18:31:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:571
2024-09-26 18:31:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:572
2024-09-26 18:31:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:573
2024-09-26 18:31:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:574
2024-09-26 18:31:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:575
2024-09-26 18:31:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:576
2024-09-26 18:31:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:577
2024-09-26 18:31:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:578
2024-09-26 18:31:42 [scrapy.core.engine] INFO: Closing spider (finished)
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] ------------------------------ MONITORS ------------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Extracted Items Monitor/test_stat_monitor... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Error Count Monitor/test_stat_monitor... SKIPPED (Unable to find 'log_count/ERROR' in job stats.)
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Warning Count Monitor/test_stat_monitor... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Unwanted HTTP codes monitor/Should not hit the limit of unwanted http status... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Field Coverage Monitor/test_check_if_field_coverage_rules_are_met... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Retry Count monitor/Should not hit the limit of requests that reached the maximum retry amount... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Downloader Exceptions monitor/test_stat_monitor... SKIPPED (Unable to find 'downloader/exception_count' in job stats.)
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Successful Requests monitor/Should have at least the minimum number of successful requests... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] Total Requests monitor/Should not hit the total limit of requests... OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] ----------------------------------------------------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] 11 monitors in 0.002s
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] OK (skipped=2)
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] -------------------------- FINISHED ACTIONS --------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] ----------------------------------------------------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] 0 actions in 0.000s
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] --------------------------- PASSED ACTIONS ---------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] ----------------------------------------------------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] 0 actions in 0.000s
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] OK
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] --------------------------- FAILED ACTIONS ---------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] ----------------------------------------------------------------------
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] 0 actions in 0.000s
2024-09-26 18:31:42 [lowes] INFO: [Spidermon] OK
2024-09-26 18:31:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (1117 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:579
2024-09-26 18:31:42 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 420345,
 'downloader/request_count': 185,
 'downloader/request_method_count/GET': 185,
 'downloader/response_bytes': 267025079,
 'downloader/response_count': 185,
 'downloader/response_status_count/200': 185,
 'elapsed_time_seconds': 829.707193,
 'feedexport/success_count/BigQueryFeedStorage': 579,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2024, 9, 26, 18, 31, 42, 758488, tzinfo=datetime.timezone.utc),
 'httpcompression/response_bytes': 1636462281,
 'httpcompression/response_count': 185,
 'item_dropped_count': 1,
 'item_dropped_reasons_count/DropItem': 1,
 'item_scraped_count': 1446117,
 'log_count/INFO': 630,
 'log_count/WARNING': 1,
 'memusage/max': 2717200384,
 'memusage/startup': 113545216,
 'poet/injector/catalog_discovery.pages.lowes.LowesSitemapPageObject': 185,
 'request_depth_max': 1,
 'response_received_count': 185,
 'scheduler/dequeued': 185,
 'scheduler/dequeued/memory': 185,
 'scheduler/enqueued': 185,
 'scheduler/enqueued/memory': 185,
 'spidermon/validation/fields': 5784472,
 'spidermon/validation/fields/errors': 1,
 'spidermon/validation/fields/errors/invalid_uri': 1,
 'spidermon/validation/fields/errors/invalid_uri/url': 1,
 'spidermon/validation/items': 1446118,
 'spidermon/validation/items/dropped': 1,
 'spidermon/validation/items/errors': 1,
 'spidermon/validation/validators': 1,
 'spidermon/validation/validators/item/jsonschema': True,
 'spidermon_field_coverage/dict/name': 1.0,
 'spidermon_field_coverage/dict/sku': 1.0,
 'spidermon_field_coverage/dict/supplier': 1.0,
 'spidermon_field_coverage/dict/url': 1.0,
 'spidermon_item_scraped_count': 1446117,
 'spidermon_item_scraped_count/dict': 1446117,
 'spidermon_item_scraped_count/dict/name': 1446117,
 'spidermon_item_scraped_count/dict/sku': 1446117,
 'spidermon_item_scraped_count/dict/supplier': 1446117,
 'spidermon_item_scraped_count/dict/url': 1446117,
 'start_time': datetime.datetime(2024, 9, 26, 18, 17, 53, 51295, tzinfo=datetime.timezone.utc)}
2024-09-26 18:31:42 [scrapy.core.engine] INFO: Spider closed (finished)