2024-09-18 23:28:24 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: catalog_discovery) 2024-09-18 23:28:24 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.9 (main, Aug 13 2024, 01:19:58) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-6.4.10-dirty-x86_64-with-glibc2.36 2024-09-18 23:28:25 [scrapy.addons] INFO: Enabled addons: [] 2024-09-18 23:28:25 [scrapy.extensions.telnet] INFO: Telnet Password: 35f7097f0e5be52d 2024-09-18 23:28:25 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats', 'spidermon.contrib.scrapy.extensions.Spidermon'] 2024-09-18 23:28:25 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'catalog_discovery', 'CONCURRENT_ITEMS': 1000, 'CONCURRENT_REQUESTS': 32, 'FEED_EXPORT_ENCODING': 'utf-8', 'LOG_FILE': '/var/lib/scrapyd/logs/catalog_discovery/home_depot/b1016192761511efb4924200a9fe0102.log', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'catalog_discovery.spiders', 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_poet.ScrapyPoetRequestFingerprinter', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'RETRY_TIMES': 5, 'SPIDER_MODULES': ['catalog_discovery.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor', 'USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:125.0) ' 'Gecko/20100101 Firefox/125.0'} 2024-09-18 23:28:25 [scrapy_poet.injection] INFO: Loading providers: [, , , , , , ] 2024-09-18 23:28:25 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy_poet.InjectionMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy_poet.DownloaderStatsMiddleware'] 2024-09-18 23:28:25 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy_poet.RetryMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2024-09-18 23:28:25 [scrapy.middleware] INFO: Enabled item pipelines: ['scraping_utils.pipelines.DuplicatesFilterPipeline', 'scraping_utils.pipelines.AttachSupplierPipeline', 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline'] 2024-09-18 23:28:25 [scrapy.core.engine] INFO: Spider opened 2024-09-18 23:28:25 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2024-09-18 23:28:25 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6024 2024-09-18 23:28:27 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.homedepot.com/sitemap/P/PIPs/25S.xml>: HTTP status code is not handled or not allowed 2024-09-18 23:28:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:1 2024-09-18 23:28:37 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Patchwork-60-in-L-x-19-in-W-Cotton-Valance-in-Charcoal-Grey-Khaki-Dark-Crème-Black-56758/323922365', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:28:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:2 2024-09-18 23:28:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Armstrong-Take-Home-Sample-ÿStylistik-II-Black-Gloss-Peel-and-Stick-Vinyl-Tile-Flooring-5-in-x-7-in-AR-115250/317157872', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:28:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:3 2024-09-18 23:28:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:4 2024-09-18 23:28:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:5 2024-09-18 23:28:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:6 2024-09-18 23:28:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:7 2024-09-18 23:28:47 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-Stripe-36-in-W-x-63-in-L-Light-Filtering-Prairie-Window-Curtain-Panel-in-Crème-Nickel-Pair-69968/324468872', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:28:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:8 2024-09-18 23:28:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:9 2024-09-18 23:28:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:10 2024-09-18 23:28:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:11 2024-09-18 23:28:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:12 2024-09-18 23:28:55 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Solar-Outdoor-Lights-Extra-Bright-Solar-Outdoor-Torch-Lights-for-Garden-Décor-12-Pack-B0B8CRZDDW/326472511', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:28:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:13 2024-09-18 23:28:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:14 2024-09-18 23:29:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:15 2024-09-18 23:29:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:16 2024-09-18 23:29:02 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Brown-Traditional-Square-Vegan-Leather-Armchair-with-Nailhead-Trims-and-Button-tufted-Back-CHYZ0520-BROWN/323233538', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:17 2024-09-18 23:29:03 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Deco-Products-Take-Home-Sample-COLORS-Glue-Down-Floor-and-Wall-DIY-Old-Desert-Aged-Wood-6-in-x-6-in-ÿMulti-TonalÿLuxury-Vinyl-Plank-SDVF8/309893945', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:04 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Simple-Life-Flax-36-in-W-x-36-in-L-Light-Filtering-Tier-Window-Panel-in-Natural-Crème-Pair-45637/324886177', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:18 2024-09-18 23:29:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:19 2024-09-18 23:29:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:20 2024-09-18 23:29:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Deco-Products-Take-Home-Sample-COLORS-Glue-Down-Floor-Wall-DIY-Old-Blue-Sea-Aged-Wood-6-in-x-6-in-ÿMulti-TonalÿLuxury-Vinyl-Plank-SDVF2/304276374', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:21 2024-09-18 23:29:12 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Camel-Traditional-Square-Vegan-Leather-Armchair-with-Nailhead-Trims-and-Button-tufted-Back-CHYZ0520-CAMEL/323233445', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:22 2024-09-18 23:29:13 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/15-75-in-W-×-39-37-in-D-×-70-87-in-H-Walnut-Freestanding-Large-Kitchen-Pantry-Storage-Cabinet-with-Drawers-Shelves-SNSA22IN169/327649352', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:13 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Solar-Lights-Outdoor-Premium-99-LEDs-Solar-Torch-Light-with-Flickering-Flame-Outdoor-Décor-4-Pack-B07PXS6GKD/326472057', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:23 2024-09-18 23:29:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:24 2024-09-18 23:29:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:25 2024-09-18 23:29:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:26 2024-09-18 23:29:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:27 2024-09-18 23:29:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:28 2024-09-18 23:29:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:29 2024-09-18 23:29:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-Stripe-36-in-W-x-63-in-L-Light-Filtering-Rod-Pocket-Window-Panel-Crème-Grey-Pair-69966/323406018', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Armstrong-Take-Home-Sample-ÿStylistik-II-White-Gloss-Peel-and-Stick-Vinyl-Tile-Flooring-5-in-x-7-in-AR-1850592/317157881', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:25 [scrapy.extensions.logstats] INFO: Crawled 75 pages (at 75 pages/min), scraped 75148 items (at 75148 items/min) 2024-09-18 23:29:25 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Gather-72-in-L-x-20-in-W-Cotton-Valance-in-Dark-Crème-Coal-Black-Khaki-56765/323922153', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:30 2024-09-18 23:29:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Tea-Cabin-36-in-L-Cotton-Swag-Valance-in-Green-Navy-Crème-Pair-27085/324295857', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:31 2024-09-18 23:29:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Mustard-3-Piece-Living-Room-Set-with-Button-Tufted-SF0422-MA3288-S2-MUS/321768095', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-36-in-W-x-24-in-L-Farmhouse-Light-Filtering-Tier-Window-Panel-in-Crème-Nickel-Grey-Pair-69971/324602505', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:32 2024-09-18 23:29:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:33 2024-09-18 23:29:31 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Green-Leafy-Vines-Wallpaper-Sample-4143-34020SAM/327154364', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:32 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Black-Square-Armchair-with-Nailhead-Trims-CHYZ0521-BLACK/323234830', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:34 2024-09-18 23:29:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:35 2024-09-18 23:29:35 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Tea-Cabin-36-in-L-Cotton-Prairie-Swag-Valance-in-Green-Navy-Crème-Pair-8270/324295855', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:36 2024-09-18 23:29:37 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Nicole-Miller-Textured-Matelassé-Dove-Grey-Abstract-Light-Filtering-Hidden-Tab-Rod-Pocket-Curtain-50-in-W-x-84-in-L-Set-of-2-EN7023-03-2-84H/310453436', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:37 2024-09-18 23:29:38 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Forest-Chinoiserie-Antique-Crème-Matte-Non-Woven-Removable-Paste-the-Wall-Wallpaper-118558/323619750', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:38 2024-09-18 23:29:41 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NICOLE-MILLER-NEW-YORK-Textured-Matelassé-Blush-Abstract-Light-Filtering-Hidden-Tab-Rod-Pocket-Curtain-50-in-W-x-108-in-L-Set-of-2-YC013585DSNMF1-A261/318757154', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:39 2024-09-18 23:29:41 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Grey-Leafy-Vines-Non-Pasted-Non-Woven-Paper-Wallpaper-4143-34016/327154438', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:40 2024-09-18 23:29:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:41 2024-09-18 23:29:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:42 2024-09-18 23:29:47 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-Stripe-40-in-W-x-84-in-L-Light-Filtering-Rod-Pocket-Window-Panel-Crème-Grey-Pair-69965/323406020', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:43 2024-09-18 23:29:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:44 2024-09-18 23:29:49 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Layered-60-in-L-x-20-in-W-Cotton-Valance-in-Country-Red-Dark-Tan-Dark-Crème-51965/323922348', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:50 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-Stripe-36-in-L-Swag-Valance-in-Crème-Nickel-Pair-69970/324295896', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:45 2024-09-18 23:29:53 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Simple-Life-Flax-40-in-W-x-84-in-L-Ruffled-Light-Filtering-Rod-Pocket-Window-Panel-Natural-Crème-Pair-51351/323480833', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:46 2024-09-18 23:29:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:47 2024-09-18 23:29:55 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Navy-Traditional-Square-Vegan-Leather-Armchair-with-Nailhead-Trims-and-Button-tufted-Back-CHYZ0520-NAVY/323233429', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:48 2024-09-18 23:29:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:49 2024-09-18 23:29:58 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Ivory-3-Piece-Living-Room-Set-with-Button-Tufted-SF0422-MA3288-S2-IVO/321768105', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:58 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Admired-By-Nature-Admired-By-Nature-Floating-Shelves-Wall-Mount-Home-Décor-ABN5E193-NTRL/328472006', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:29:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:50 2024-09-18 23:29:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:51 2024-09-18 23:30:00 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grain-Sack-36-in-W-x-63-in-L-Light-Filtering-Prairie-Window-Curtain-Panel-in-Crème-Aqua-Pair-70119/324468472', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:52 2024-09-18 23:30:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:53 2024-09-18 23:30:03 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Cecília-Mustard-Armchair-With-Solid-Wood-Legs-CHM0584-MUSTARD/322864220', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:54 2024-09-18 23:30:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Patchwork-72-in-L-x-19-in-W-Cotton-Valance-in-Charcoal-Grey-Khaki-Dark-Crème-Black-38040/323922372', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NICOLE-MILLER-NEW-YORK-Textured-Matelassé-Blush-Abstract-Light-Filtering-Hidden-Tab-Rod-Pocket-Curtain-50-in-W-x-96-in-L-Set-of-2-EN7024-04-2-96H/310453526', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:55 2024-09-18 23:30:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:56 2024-09-18 23:30:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:57 2024-09-18 23:30:11 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Brown-Traditional-Square-Vegan-Leather-Armchair-with-Nailhead-Trims-and-Button-tufted-Back-Set-of-2-CHYZ0520-BROWN-S2/323233467', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:58 2024-09-18 23:30:12 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Frayed-Lattice-36-in-W-x-24-in-L-Light-Filtering-Tier-Window-Panel-in-Demin-Grey-Black-Crème-Pair-80513/324602495', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:59 2024-09-18 23:30:14 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Fushia-3-Piece-Living-Room-Set-with-Button-Tufted-SF0422-MA3288-S2-FUS/321768119', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Simple-Life-Ruffled-Flax-36-in-W-x-36-in-L-Light-Filtering-Tier-Window-Panel-in-Natural-Crème-Pair-51969/324886212', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:60 2024-09-18 23:30:16 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/21-in-W-x47-in-H-Traditional-Decor-Style-Rectangle-Antique-Glass-Wall-Mirror-Wall-Décor-for-Bathroom-Entryway-WQQ-1028/329036805', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:61 2024-09-18 23:30:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:62 2024-09-18 23:30:19 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Ticking-Stripe-72-in-L-x-16-in-W-Cotton-Valance-in-Coal-Black-Dark-Crème-51930/323922340', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:19 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Simple-Life-Flax-36-in-W-x-63-in-L-Ruffled-Light-Filtering-Rod-Pocket-Window-Panel-Natural-Crème-Pair-51352/323480834', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:19 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Ticking-Stripe-40-in-W-x-72-in-L-Light-Filtering-Rod-Pocket-French-Door-Window-Panel-in-Crème-Charcoal-51301/324204796', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:63 2024-09-18 23:30:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:64 2024-09-18 23:30:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:65 2024-09-18 23:30:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:66 2024-09-18 23:30:23 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Simple-Life-Flax-40-in-W-x-84-in-L-Light-Filtering-Rod-Pocket-Window-Panel-Natural-Crème-Pair-45631/323480289', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:23 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NICOLE-MILLER-NEW-YORK-Textured-Matelassé-Snowflake-Abstract-Light-Filtering-Hidden-Tab-Rod-Pocket-Curtain-50-in-W-x-84-in-L-Set-of-2-EN7023-02-2-84H/310453419', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:67 2024-09-18 23:30:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Botanical-Bee-Crème-Matte-Non-Woven-Removable-Paste-the-Wall-Wallpaper-118544/323601295', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Patchwork-72-in-L-x-19-in-W-Cotton-Valance-in-Country-Red-Dark-Tan-Dark-Crème-51964/323922375', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:68 2024-09-18 23:30:25 [scrapy.extensions.logstats] INFO: Crawled 75 pages (at 0 pages/min), scraped 171258 items (at 96110 items/min) 2024-09-18 23:30:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:69 2024-09-18 23:30:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Ticking-Stripe-36-in-W-x-24-in-L-Light-Filtering-Tier-Window-Panel-in-Dark-Crème-Coal-Black-Pair-51927/324812560', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Cecília-Charcoal-Armchair-With-Solid-Wood-Legs-Set-of-2-CHM0584-CHARCOAL-S2/322865690', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:70 2024-09-18 23:30:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-Stripe-72-in-L-x-16-in-W-Cotton-Valance-in-Crème-Nickel-Grey-69974/323922234', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:71 2024-09-18 23:30:29 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/AndMakers-Liverpool-46-in-W-×-36-in-H-Rectangle-Wood-Frame-Black-Dresser-Mirror-PF-G01600-M/327132262', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:72 2024-09-18 23:30:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:73 2024-09-18 23:30:31 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Simple-Life-Ruffled-Flax-36-in-W-x-24-in-L-Light-Filtering-Tier-Window-Panel-in-Natural-Crème-Pair-51968/324886211', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:74 2024-09-18 23:30:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:75 2024-09-18 23:30:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-Stripe-36-in-W-x-84-in-L-Light-Filtering-Prairie-Window-Curtain-Panel-in-Crème-Nickel-Pair-69967/324468871', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NICOLE-MILLER-NEW-YORK-Textured-Matelassé-Snowflake-Abstract-Light-Filtering-Hidden-Tab-Rod-Pocket-Curtain-50-in-W-x-108-in-L-Set-of-2-YC013585DSNMF1-A217/318757345', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/81-in-×-19-in-L-Shaped-Brown-3-Drawer-and-Shelves-Convertible-Home-Office-Computer-Desk-with-Charging-Station-TOPB007764/330056222', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Sawyer-Mill-Patchwork-60-in-L-x-19-in-W-Cotton-Valance-in-Country-Red-Dark-Tan-Dark-Crème-51963/323922370', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:76 2024-09-18 23:30:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:77 2024-09-18 23:30:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:78 2024-09-18 23:30:37 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Yellow-Drömma-Butter-Songbirds-and-Sunflowers-Paper-Non-Pasted-Non-Woven-Matte-Wallpaper-4111-63010/322489143', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:38 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Wilder-36-in-W-x-63-in-L-Floral-Light-Filtering-Rod-Pocket-Window-Panel-Crème-Blue-Green-Pink-Gold-Pair-81281/323480033', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:79 2024-09-18 23:30:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Grace-Grain-Sack-36-in-W-x-36-in-L-Farmhouse-Light-Filtering-Tier-Window-Panel-in-Crème-Nickel-Grey-Pair-69972/324602507', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:80 2024-09-18 23:30:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:81 2024-09-18 23:30:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:82 2024-09-18 23:30:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:83 2024-09-18 23:30:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Cecília-Ivory-Armchair-With-Solid-Wood-Legs-Set-of-2-CHM0584-IVORY-S2/322865704', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:84 2024-09-18 23:30:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/White-Wooden-Shoes-Storage-Cabinet-with-Adjustable-Shelves-and-2-Drawers-31-5-in-W-×-39-4-in-H-LBB-KF260056-01/323767895', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:85 2024-09-18 23:30:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:86 2024-09-18 23:30:47 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Cecília-Navy-Armchair-with-Solid-Wood-Legs-CHM0584-NAVY/322864207', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:30:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:87 2024-09-18 23:30:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:88 2024-09-18 23:30:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:89 2024-09-18 23:30:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:90 2024-09-18 23:30:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:91 2024-09-18 23:30:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:92 2024-09-18 23:30:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:93 2024-09-18 23:30:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:94 2024-09-18 23:30:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:95 2024-09-18 23:30:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:96 2024-09-18 23:30:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:97 2024-09-18 23:31:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:98 2024-09-18 23:31:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:99 2024-09-18 23:31:02 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Grey-Square-Armchair-with-Nailhead-Trims-CHYZ0521-GREY/323235038', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:100 2024-09-18 23:31:03 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Camel-Traditional-Square-Vegan-Leather-Armchair-with-Nailhead-Trims-and-Button-tufted-Back-Set-of-2-CHYZ0520-CAMEL-S2/323233476', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:101 2024-09-18 23:31:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:102 2024-09-18 23:31:06 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Navy-Traditional-Square-Vegan-Leather-Armchair-with-Nailhead-Trims-and-Button-tufted-Back-Set-of-2-CHYZ0520-NAVY-S2/323233570', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:103 2024-09-18 23:31:07 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Cecília-Ivory-Armchair-With-Solid-Wood-Legs-CHM0584-IVORY/322864336', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:104 2024-09-18 23:31:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Red-3-Piece-Living-Room-Set-with-Button-Tufted-SF0422-MA3288-S2-RED/321768135', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Grey-Armchair-with-Nailhead-Trims-Set-of-2-CHYZ0521-GREY-S2/323235068', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:105 2024-09-18 23:31:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:106 2024-09-18 23:31:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:107 2024-09-18 23:31:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:108 2024-09-18 23:31:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:109 2024-09-18 23:31:14 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Benjara-Brown-Wood-Top-16-in-Sideboard-with-Cane-Façade-Door-Panels-BM295807/328229596', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:110 2024-09-18 23:31:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:111 2024-09-18 23:31:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:112 2024-09-18 23:31:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:113 2024-09-18 23:31:19 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Cecília-Charcoal-Armchair-With-Solid-Wood-Legs-CHM0584-CHARCOAL/322864309', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:114 2024-09-18 23:31:20 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Benjara-Black-and-Brown-Wood-Top-16-in-Sideboard-with-Cane-Façade-Door-Panels-BM295805/328229595', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:115 2024-09-18 23:31:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:116 2024-09-18 23:31:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Cecília-Grey-Armchair-with-Solid-Wood-Legs-CHM0584-GREY/322864211', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:117 2024-09-18 23:31:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Navy-3-Piece-Living-Room-Set-with-Button-tufted-SF0422-MA3288-S2-NAV/321768148', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:118 2024-09-18 23:31:25 [scrapy.extensions.logstats] INFO: Crawled 75 pages (at 0 pages/min), scraped 297879 items (at 126621 items/min) 2024-09-18 23:31:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:119 2024-09-18 23:31:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:120 2024-09-18 23:31:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:121 2024-09-18 23:31:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:122 2024-09-18 23:31:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:123 2024-09-18 23:31:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Mina-Victory-Inspire-Me-Home-Décor-Black-20-in-x-20-in-Throw-Pillow-072919/322138550', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:124 2024-09-18 23:31:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:125 2024-09-18 23:31:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:126 2024-09-18 23:31:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:127 2024-09-18 23:31:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Light-Grey-Leafy-Vines-Non-Pasted-Non-Woven-Paper-Wallpaper-4143-34017/327154439', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Green-Drömma-Sage-Songbirds-and-Sunflowers-Wallpaper-Sample-4111-63009SAM/322509969', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:128 2024-09-18 23:31:40 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/INNOVERA-DÉCOR-BY-PALRAM-15-7-in-x24-4-in-Tongue-Groove-Decorative-PVC-Bathroom-and-Shower-Wall-Tiles-in-Rustic-Concrete-Light-Gray-8-Piece-706010/313348296', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:129 2024-09-18 23:31:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NOTRE-DAME-DESIGN-Asher-36-in-H-x-36-in-W-Modern-Round-Black-Framed-Décor-Mirror-NDD22M025/323207462', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:130 2024-09-18 23:31:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:131 2024-09-18 23:31:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:132 2024-09-18 23:31:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:133 2024-09-18 23:31:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:134 2024-09-18 23:31:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:135 2024-09-18 23:31:49 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Uniquewise-Modern-Farmhouse-Home-Décor-Accents-Boho-Vases-for-Table-Decor-Black-Ceramic-Centerpiece-Vase-for-Home-Decor-Set-of-3-QI004358-3/320243242', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:49 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Becky-Cameron-Light-Gray-Seed-Stitch-Knit-Acrylic-20-in-x-20-in-Décor-Throw-Pillow-INSRT-TPC-SKN20-LGRAY/329160301', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:136 2024-09-18 23:31:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:137 2024-09-18 23:31:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/48-in-Artificial-Cedar-Topiary-Trees-in-Black-Planter-Perfect-Patio-Entryway-Décor-2-Pack-FL02314-2/326594481', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:138 2024-09-18 23:31:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:139 2024-09-18 23:31:54 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Uniquewise-Tall-Modern-Decorative-Floor-Vase-Handmade-Natural-Bamboo-Finish-Contemporary-Home-Décor-Handcrafted-Bamboo-Large-QI003592-L/312276139', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:140 2024-09-18 23:31:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:141 2024-09-18 23:31:56 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Home-Decorators-Collection-Plush-Red-Plaid-Sherpa-Throw-Blanket-ST50×70PBPHR/316976426', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:142 2024-09-18 23:31:58 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/TOLEDO-Antique-Brass-Privacy-Málaga-Knob-CV1920MAUS5/202530837', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:31:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:143 2024-09-18 23:31:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:144 2024-09-18 23:32:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:145 2024-09-18 23:32:00 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Yellow-Drömma-Coral-Songbirds-and-Sunflowers-Paper-Non-Pasted-Non-Woven-Matte-Wallpaper-4111-63008/322489136', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:146 2024-09-18 23:32:03 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/TOLEDO-Toledo-stainless-steel-Málaga-passage-Hall-Closet-Door-Knob-CV1910MAUS32D/307377043', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:147 2024-09-18 23:32:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:148 2024-09-18 23:32:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:149 2024-09-18 23:32:06 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Subway-12-in-×-4-in-×-0-12-in-Black-Peel-and-Stick-backsplash-Stone-Composite-Wall-Tile-10-sq-ft-Case-100300-BLK-30/328031122', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:150 2024-09-18 23:32:06 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Midnight-Beasts-Blush-Crème-Matte-Non-Woven-Removable-Paste-the-Wall-Wallpaper-118565/323601316', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:151 2024-09-18 23:32:07 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NuWallpaper-Grey-Brick-Façade-Gray-Wallpaper-Sample-NU2236SAM/301900858', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Richelieu-Hardware-Champs-Élysees-Collection-2-3-4-in-69-mm-x-13-16-in-20-mm-Matte-Black-Traditional-Cabinet-Knob-BP10569900/204695503', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Notre-Dame-Façade-by-Marmont-Hill-Framed-Architecture-Art-Print-30-in-x-45-in-LITI5002BFPFL45/324657669', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:152 2024-09-18 23:32:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:153 2024-09-18 23:32:10 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/York-Wallcoverings-Cream-Café-Society-Paper-Wallpaper-Matte-27-in-by-27-ft-CI2364/325573414', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:154 2024-09-18 23:32:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:155 2024-09-18 23:32:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:156 2024-09-18 23:32:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:157 2024-09-18 23:32:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:158 2024-09-18 23:32:16 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Finders-Keepers-Natural-Crème-Steel-Grey-Farmhouse-Chicken-Silhouette-6-in-x-6-in-Throw-Pillow-84348/328620215', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:159 2024-09-18 23:32:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:160 2024-09-18 23:32:18 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Pink-3-Piece-Living-Room-Set-with-Button-Tufted-SF0422-MA3288-S2-PIN/321768143', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:161 2024-09-18 23:32:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:162 2024-09-18 23:32:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:163 2024-09-18 23:32:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:164 2024-09-18 23:32:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:165 2024-09-18 23:32:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:166 2024-09-18 23:32:25 [scrapy.extensions.logstats] INFO: Crawled 107 pages (at 32 pages/min), scraped 416467 items (at 118588 items/min) 2024-09-18 23:32:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:167 2024-09-18 23:32:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Storied-Home-Ombré-Ceramic-Vase-with-Reactive-Glaze-6-25-in-in-Black-and-White-AH0405/323770337', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:168 2024-09-18 23:32:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Home-Decorators-Collection-Plush-Washed-Denim-Sherpa-Throw-Blanket-ST50×60B/313099557', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:169 2024-09-18 23:32:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:170 2024-09-18 23:32:30 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Modern-Farmhouse-Home-Décor-Accents-Boho-Vases-for-Decor-Housewarming-Gifts-Green-10-in-QI004055-DGN/318183267', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:31 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/StyleWell-Charcoal-Gray-15-lb-Weighted-Blanket-WB-50×70-15C/320662991', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:171 2024-09-18 23:32:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:172 2024-09-18 23:32:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:173 2024-09-18 23:32:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:174 2024-09-18 23:32:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:175 2024-09-18 23:32:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Artificial-White-Berry-Stem-Picks-Wire-Branch-Sprays-for-Xmas-Tree-Décor-24-Pack-AMZ0401WH-1BG/325019599', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:176 2024-09-18 23:32:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:177 2024-09-18 23:32:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Modern-Farmhouse-Home-Décor-Accents-Vases-for-Table-Decor-Housewarming-Gift-8-in-Yellow-Mustard-QI004055-MD/318183277', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:178 2024-09-18 23:32:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:179 2024-09-18 23:32:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NOTRE-DAME-DESIGN-Beccy-Pill-40-in-H-x-26-in-W-Modern-Rectangular-Brass-Framed-Décor-Mirror-NDD22M024/323207460', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:180 2024-09-18 23:32:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:181 2024-09-18 23:32:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:182 2024-09-18 23:32:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:183 2024-09-18 23:32:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:184 2024-09-18 23:32:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:185 2024-09-18 23:32:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:186 2024-09-18 23:32:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:187 2024-09-18 23:32:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:188 2024-09-18 23:32:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Fields-Edge-Floral-Antique-Crème-Matte-Non-Woven-Removable-Paste-The-Wall-Wallpaper-Sample-11856994/323793215', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:189 2024-09-18 23:32:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:190 2024-09-18 23:32:54 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/9-1-2-in-diameter-x-7-in-Height-Blue-Glass-Candle-Holder-Stand-for-Wedding-Dining-Party-Home-Décor-16021/325106008', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:191 2024-09-18 23:32:55 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/4-ft-Artificial-Areca-Palm-Tree-Tall-Fake-Palm-Tree-Décor-2-Pack-B09CTNFL77/324952021', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:56 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/AndMakers-Albany-48-in-W-×-38-in-H-Rectangle-Wood-Frame-Washed-Gray-Dresser-Mirror-PF-G02200-M/327132264', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:192 2024-09-18 23:32:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:193 2024-09-18 23:32:58 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Grey-3-Piece-Living-Room-Set-with-Button-Tufted-SF0422-MA3288-S2-GRE/321768153', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:32:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:194 2024-09-18 23:33:00 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Mina-Victory-Inspire-Me-Home-Décor-Ivory-18-in-x-18-in-Throw-Pillow-075439/322138608', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:195 2024-09-18 23:33:00 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Stratton-Home-Decor-Black-Metal-Medallion-Wall-Décor-Set-of-3-S50148/323937371', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:196 2024-09-18 23:33:02 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/StyleWell-Charcoal-Gray-12-lb-Weighted-Blanket-WB-50×70-12C/316976418', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:02 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Niccolò-Black-3-Piece-Living-Room-Set-with-Button-Tufted-SF0422-MA3288-S2-BLA/321768099', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:197 2024-09-18 23:33:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:198 2024-09-18 23:33:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:199 2024-09-18 23:33:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:200 2024-09-18 23:33:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:201 2024-09-18 23:33:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JAYDEN-CREATION-Vinícius-Navy-Armchair-with-Nailhead-Trims-Set-of-2-CHYZ0521-NAVY-S2/323235011', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:202 2024-09-18 23:33:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NOTRE-DAME-DESIGN-Chris-24-in-W-x-36-in-H-Modern-Arch-RectangularGold-Framed-Décor-Mirror-NDD22M032/323207494', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Grey-Drömma-Light-Grey-Songbirds-and-Sunflowers-Wallpaper-Sample-4111-63011SAM/322509989', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:203 2024-09-18 23:33:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:204 2024-09-18 23:33:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:205 2024-09-18 23:33:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:206 2024-09-18 23:33:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:207 2024-09-18 23:33:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/2-Peice-29-92-in-Faux-Artificial-Ficus-Tree-in-Pot-Artificial-Plant-for-Indoor-Outdoor-Minimalist-Home-Office-Room-Décor-YM-73Tree/326271144', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Plush-Putty-Sherpa-Throw-Blanket-ST50-×60K/313099552', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:208 2024-09-18 23:33:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:209 2024-09-18 23:33:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:210 2024-09-18 23:33:18 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Mina-Victory-Inspire-Me-Home-Décor-Navy-18-in-x-18-in-Throw-Pillow-075446/322138582', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:211 2024-09-18 23:33:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:212 2024-09-18 23:33:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:213 2024-09-18 23:33:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Home-Decorators-Collection-Plush-Black-Buffalo-Check-Sherpa-Throw-Blanket-ST50×70PBBC/316976422', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:214 2024-09-18 23:33:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:215 2024-09-18 23:33:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:216 2024-09-18 23:33:25 [scrapy.extensions.logstats] INFO: Crawled 107 pages (at 0 pages/min), scraped 541284 items (at 124817 items/min) 2024-09-18 23:33:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:217 2024-09-18 23:33:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:218 2024-09-18 23:33:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:219 2024-09-18 23:33:30 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Plush-Gray-Sherpa-Throw-Blanket-ST50×60G/313099553', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:30 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Modern-Farmhouse-Home-Décor-Accents-5-in-Vase-Vases-for-Decor-Housewarming-Gift-Table-Vase-Red-QI004055-RD/318183289', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:220 2024-09-18 23:33:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:221 2024-09-18 23:33:31 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Uniquewise-Modern-Farmhouse-Home-Décor-Accents-Boho-Vases-for-Table-Decor-Black-Ceramic-Centerpiece-Vase-for-Home-Decor-Large-QI004358-L/320243238', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:222 2024-09-18 23:33:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:223 2024-09-18 23:33:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:224 2024-09-18 23:33:35 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Handford-Garden-Birds-Antique-Crème-Matte-Non-Woven-Removable-Paste-the-Wall-Wallpaper-118562/323601327', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:225 2024-09-18 23:33:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:226 2024-09-18 23:33:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:227 2024-09-18 23:33:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:228 2024-09-18 23:33:40 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Richelieu-Hardware-Rosemère-Collection-1-3-4-in-44-mm-Classic-Brushed-Nickel-Rectangle-Cabinet-Knob-BP76544195/204078101', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:229 2024-09-18 23:33:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:230 2024-09-18 23:33:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Becky-Cameron-Washed-and-Distressed-Cotton-20-in-x-20-in-Décor-Throw-Pillow-INSRT-TPC20-OLIVE/329160314', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:231 2024-09-18 23:33:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:232 2024-09-18 23:33:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Sketchy-Dogs-Crème-Matte-Non-Woven-Removable-Paste-the-Wall-Wallpaper-118559/323601250', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/INNOVERA-DÉCOR-BY-PALRAM-15-7-in-x-24-4-in-Tongue-Groove-Decorative-PVC-Bathroom-and-Shower-Wall-Tiles-in-Carrara-Marble-White-8-Piece-706007/313348307', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:45 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Home-Decorators-Collection-Plush-Leopard-Sherpa-Throw-Blanket-ST50×70PL/316976427', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:233 2024-09-18 23:33:47 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/York-Wallcoverings-Grey-Café-Society-Paper-Unpasted-Matte-Wallpaper-27-in-x-27-ft-CI2363/325573413', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:234 2024-09-18 23:33:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:235 2024-09-18 23:33:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:236 2024-09-18 23:33:49 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Richelieu-Hardware-Champs-Élysees-Collection-2-3-16-in-55-mm-x-13-16-in-20-mm-Oil-Rubbed-Bronze-Traditional-Cabinet-Knob-BP10555ORB/205417455', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:237 2024-09-18 23:33:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:238 2024-09-18 23:33:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Moss-Leafy-Vines-Non-Pasted-Non-Woven-Paper-Wallpaper-4143-34020/327154441', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:239 2024-09-18 23:33:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:240 2024-09-18 23:33:55 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/INNOVERA-DÉCOR-BY-PALRAM-18-5-in-x-24-3-in-Empire-Decorative-3D-PVC-Backsplash-Panels-in-Bronze-30-Pieces-704550/321723158', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:241 2024-09-18 23:33:55 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Finders-Keepers-Natural-Crème-Charcoal-Grey-Farmhouse-Kindness-7-in-x-13-in-Throw-Pillow-84346/328620213', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:242 2024-09-18 23:33:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:243 2024-09-18 23:33:58 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Boho-Bloom-Crème-Matte-Non-Woven-Removable-Paste-the-Wall-Wallpaper-118561/323601324', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:33:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:244 2024-09-18 23:34:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:245 2024-09-18 23:34:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:246 2024-09-18 23:34:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:247 2024-09-18 23:34:04 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Home-Decorators-Collection-Plush-Blue-Buffalo-Check-Sherpa-Throw-Blanket-ST50×60PB/320662879', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:248 2024-09-18 23:34:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:249 2024-09-18 23:34:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:250 2024-09-18 23:34:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:251 2024-09-18 23:34:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:252 2024-09-18 23:34:10 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Forrester-Indoor-Outdoor-8-in-W-x-24-in-L-Red-Green-Crème-Oval-Stripe-PET-Table-Runner-84123/327610448', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:253 2024-09-18 23:34:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:254 2024-09-18 23:34:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:255 2024-09-18 23:34:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:256 2024-09-18 23:34:17 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Finders-Keepers-Natural-Crème-Steel-Grey-Farmhouse-Cow-Silhouette-6-in-x-6-in-Throw-Pillow-84349/328620216', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:257 2024-09-18 23:34:19 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/StyleWell-Puffer-with-Fleece-Dark-Blue-Solid-Gray-Nylon-Polyester-Throw-Blanket-PSFT50×60/324404878', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:258 2024-09-18 23:34:20 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Forrester-Indoor-Outdoor-12-in-W-x-48-in-L-Red-Green-Crème-Oval-Stripe-PET-Table-Runner-84125/327610447', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:259 2024-09-18 23:34:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Richelieu-Hardware-Champs-Élysees-Collection-2-3-16-in-55-mm-x-13-16-in-20-mm-Matte-Black-Traditional-Cabinet-Knob-BP10555900/204695500', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/AndMakers-Franklin-39-in-W-×-35-in-H-Rectangle-Wood-Frame-Brown-Dresser-Mirror-PF-G01900-M/327132263', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:260 2024-09-18 23:34:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:261 2024-09-18 23:34:25 [scrapy.extensions.logstats] INFO: Crawled 107 pages (at 0 pages/min), scraped 653569 items (at 112285 items/min) 2024-09-18 23:34:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:262 2024-09-18 23:34:27 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Becky-Cameron-Eucalyptus-Seed-Stitch-Knit-Acrylic-20-in-x-20-in-Décor-Throw-Pillow-INSRT-TPC-SKN20-EUCALYPTUS/329160299', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:27 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Home-Decorators-Collection-Plush-Red-Buffalo-Check-Sherpa-Throw-Blanket-ST50×70PRBC/316976421', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:263 2024-09-18 23:34:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:264 2024-09-18 23:34:30 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Becky-Cameron-Gray-Cozy-Chunky-Knit-Acrylic-20-in-x-20-in-Décor-Throw-Pillow-INSRT-TPC-CHKN20-GRAY/329160266', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:265 2024-09-18 23:34:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:266 2024-09-18 23:34:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:267 2024-09-18 23:34:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Matelassé-Organic-Cotton-Twin-XL-Blanket-in-Taupe-BK3303TPTX-4512/328108642', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:268 2024-09-18 23:34:38 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Notre-Dame-Façade-by-Marmont-Hill-Framed-Architecture-Art-Print-24-in-x-36-in-LITI5002BFPFL36/324657667', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:269 2024-09-18 23:34:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:270 2024-09-18 23:34:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:271 2024-09-18 23:34:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:272 2024-09-18 23:34:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Mina-Victory-Inspire-Me-Home-Décor-Gray-18-in-x-18-in-Throw-Pillow-075422/322138575', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:46 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/York-Wallcoverings-Cream-Café-Society-Paper-Unpasted-Matte-Wallpaper-27-in-x-27-ft-CI2361/325573411', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:273 2024-09-18 23:34:46 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/StyleWell-Packable-Navy-and-Khaki-Plaid-Fleece-Outdoor-Throw-Blanket-PWF50×60MRP/320662877', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:274 2024-09-18 23:34:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:275 2024-09-18 23:34:49 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/York-Wallcoverings-Dark-Grey-Café-Society-Paper-Unpasted-Matte-Wallpaper-27-in-x-27-ft-CI2365/325573415', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:276 2024-09-18 23:34:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:277 2024-09-18 23:34:54 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Mina-Victory-Inspire-Me-Home-Décor-Taupe-18-in-x-18-in-Throw-Pillow-075453/322138548', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:278 2024-09-18 23:34:54 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Elements-36-in-x-12-in-Metal-Geometric-Wall-Décor-Set-of-2-5282168/325389591', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:55 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Blue-Drömma-Teal-Songbirds-and-Sunflowers-Wallpaper-Sample-4111-63007SAM/322509973', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:279 2024-09-18 23:34:56 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Yellow-Drömma-Butter-Songbirds-and-Sunflowers-Wallpaper-Sample-4111-63010SAM/322509985', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:57 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Notre-Dame-Façade-by-Marmont-Hill-Framed-Architecture-Art-Print-8-in-x-12-in-LITI5002BFPFL12/324657653', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:34:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:280 2024-09-18 23:34:59 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VHC-Brands-Finders-Keepers-Natural-Crème-Steel-Grey-Farmhouse-Rooster-Silhouette-6-in-x-6-in-Throw-Pillow-84350/328620217', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:281 2024-09-18 23:35:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:282 2024-09-18 23:35:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:283 2024-09-18 23:35:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:284 2024-09-18 23:35:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Grey-Leafy-Vines-Wallpaper-Sample-4143-34016SAM/327154360', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:285 2024-09-18 23:35:07 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Yellow-Drömma-Coral-Songbirds-and-Sunflowers-Wallpaper-Sample-4111-63008SAM/322509976', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:286 2024-09-18 23:35:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:287 2024-09-18 23:35:10 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NOTRE-DAME-DESIGN-Glen-36-in-H-x-24-in-W-Modern-Rectangular-Black-Framed-Décor-Mirror-NDD22M029/323207464', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:288 2024-09-18 23:35:12 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Grey-Leafy-Vines-Wallpaper-Sample-4143-34018SAM/327154362', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:13 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/NOTRE-DAME-DESIGN-Everclear-Pill-36-in-H-x-20-in-W-Modern-Rectangular-Natural-Brown-Framed-Décor-Mirror-NDD22M001/323207455', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:13 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Notre-Dame-Façade-by-Marmont-Hill-Framed-Architecture-Art-Print-12-in-x-18-in-LITI5002BFPFL18/324657721', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:289 2024-09-18 23:35:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Taupe-Leafy-Vines-Non-Pasted-Non-Woven-Paper-Wallpaper-4143-34018/327154440', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:290 2024-09-18 23:35:16 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Notre-Dame-Façade-by-Marmont-Hill-Framed-Architecture-Art-Print-16-in-x-24-in-LITI5002BFPFL24/324657722', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:17 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/INNOVERA-DÉCOR-BY-PALRAM-18-5-in-x-24-3-in-Empire-Decorative-3D-PVC-Backsplash-Panels-in-Smoked-Pewter-30-Pieces-705812/321723278', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:291 2024-09-18 23:35:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:292 2024-09-18 23:35:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:293 2024-09-18 23:35:21 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lindlöv-Light-Grey-Leafy-Vines-Wallpaper-Sample-4143-34017SAM/327154361', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:22 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.homedepot.com/sitemap/P/PIPs/25P/025-007-0.xml>: HTTP status code is not handled or not allowed 2024-09-18 23:35:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:294 2024-09-18 23:35:25 [scrapy.extensions.logstats] INFO: Crawled 135 pages (at 28 pages/min), scraped 735209 items (at 81640 items/min) 2024-09-18 23:35:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/INNOVERA-DÉCOR-BY-PALRAM-24-3-in-x-18-5-in-Empire-Decorative-3D-PVC-Backsplash-Panels-in-Bronze-12-Pieces-706688/316856631', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/16-Piece-Ceramic-Kitchen-Cookware-Pots-and-Frying-Sauce-Sauté-Pans-Set-Caribbean-Blue-SNPH002IN441/328268318', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:295 2024-09-18 23:35:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:296 2024-09-18 23:35:32 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Croton-Petra-Indoor-Plant-in-8-75-Taupe-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-10CROPETAUPEMOD/308040177', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:297 2024-09-18 23:35:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:298 2024-09-18 23:35:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/71-3-in-H-×-63-in-L-4-Panel-Iron-Black-Room-Divider-Folding-Privacy-Screen-Wall-Partition-TOUTD712/326297692', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:299 2024-09-18 23:35:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:300 2024-09-18 23:35:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:301 2024-09-18 23:35:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:302 2024-09-18 23:35:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:303 2024-09-18 23:35:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Nostalgia-Café-Ice-12-Cup-Black-Iced-Coffee-and-Tea-Brewing-System-with-Plastic-Pitcher-NCIT3PLSBK/324567885', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:304 2024-09-18 23:35:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:305 2024-09-18 23:35:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:306 2024-09-18 23:35:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:307 2024-09-18 23:35:51 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ZUHNË-Modena-28-in-Stainless-Steel-Undermount-Kitchen-Sink-with-Accessories-16-Gauge-Modena28/314191234', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:308 2024-09-18 23:35:53 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Yucca-Cane-Indoor-Plant-in-8-75-Gray-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-10YUCTAUPEMOD/307883265', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:54 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/GreenPan-Valencia-Pro-11-in-4-5-qt-Aluminum-Ceramic-Non-Stick-Sauté-Frying-Pan-with-Handles-and-Lid-CC000671-001/326387442', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:35:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:309 2024-09-18 23:35:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:310 2024-09-18 23:35:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:311 2024-09-18 23:35:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:312 2024-09-18 23:36:00 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/71-3-in-H-×-63-in-L-4-Panel-Iron-Brown-Room-Divider-Folding-Privacy-Screen-Wall-Partition-TOUTD713/326297832', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:313 2024-09-18 23:36:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:314 2024-09-18 23:36:03 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/20-in-Peanuts-Led-3D-Prelit-Yard-Décor-Skating-Woodstock-PW-36247/325909964', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:315 2024-09-18 23:36:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:316 2024-09-18 23:36:06 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/BergHOFF-Balance-3-1-qt-Nonstick-Recycled-Aluminum-Sauté-Pan-10-25-in-with-Glass-Lid-Moonmist-3950427/328831461', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:317 2024-09-18 23:36:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:318 2024-09-18 23:36:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Vigoro-Sago-Palm-Indoor-Plant-in-6-in-White-Ribbed-Plastic-Décor-Planter-Avg-Shipping-Height-1-2-ft-Tall-2-Pack-CO-SAGO-3-VI-WH/318747128', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:10 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Glitzhome-30-in-H-Easter-Wooden-Stacked-Chicks-Porch-Décor-KD-2006700021/324284366', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:10 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Glitzhome-42-in-H-St-Patrick-s-Wooden-Gonme-Porch-Décor-GH2032700001/325009946', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:319 2024-09-18 23:36:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:320 2024-09-18 23:36:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:321 2024-09-18 23:36:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:322 2024-09-18 23:36:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:323 2024-09-18 23:36:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:324 2024-09-18 23:36:20 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Philodendron-Selloum-Indoor-Plant-in-10-in-Gray-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-CO-PS10-3-CYL/319268575', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:21 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/5-in-W-×-5-in-H-Black-Plastic-Garden-Landscape-Edging-Border-Coil-20-ft-with-6-Pieces-Stakes-TOUTD732/326308838', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:325 2024-09-18 23:36:21 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/85-6-in-×-72-in-PVC-Arched-Arbor-PT-10-00003-W/323376685', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:326 2024-09-18 23:36:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:327 2024-09-18 23:36:25 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Sago-Palm-Indoor-Plant-in-10-in-Natural-Décor-Basket-Avg-Shipping-Height-2-3-ft-Tall-CO-RV10-3-SEA/319495687', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:25 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/LYNK-PROFESSIONAL-LYNK-PROFESSIONAL-Élite-Pull-Out-Spice-Rack-Organizer-for-Cabinet-8-1-4-in-Wide-Double-Wood-Chrome-432722DS/324765408', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:25 [scrapy.extensions.logstats] INFO: Crawled 150 pages (at 15 pages/min), scraped 819226 items (at 84017 items/min) 2024-09-18 23:36:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Glitzhome-41-5-in-H-Easter-Wooden-and-Metal-Carrot-Porch-Décor-GH2006700020/325008756', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:328 2024-09-18 23:36:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/4-5-Qt-Stainless-Steel-Sauté-Pan-CW-SUTP-201/328732433', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:329 2024-09-18 23:36:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:330 2024-09-18 23:36:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:331 2024-09-18 23:36:32 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Nearly-Natural-20-in-Holiday-Merry-Christmas-Door-Wall-Hanger-with-Pinecones-and-Berries-Gingham-Plaid-Bow-Wall-Art-Décor-7148/321542121', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:33 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/16-Piece-Ceramic-Kitchen-Cookware-Pots-and-Frying-Sauce-Sauté-Pans-Set-Turquoise-SNPH002IN434/328266393', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:332 2024-09-18 23:36:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Non-Woven-Fiber-Mulch-Tree-Ring-Kit-Día-5-91-in-12-Pack-NWMTR15D12P/321295304', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Vigoro-Grower-s-Choice-Sansevieria-Indoor-Snake-Plant-in-6-in-White-Ribbed-Décor-Pot-Avg-Shipping-Height-1-2-ft-2-Pack-CO-SL07-3-VI-WH/318745813', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:333 2024-09-18 23:36:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Vigoro-Grower-s-Choice-Dracaena-Indoor-Plant-in-6-in-White-Ribbed-Plastic-Décor-Planter-Avg-Shipping-Height-1-2-ft-2-Pack-CO-JC06-3-VI-WH/318745574', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:334 2024-09-18 23:36:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:335 2024-09-18 23:36:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/BergHOFF-Balance-4-Piece-Ceramic-Ramekin-Crème-Brulé-Set-4-25-7oz-Moonbeam-3950503/328780961', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/BergHOFF-Balance-3-1-qt-Nonstick-Recycled-Aluminum-Sauté-Pan-10-25-in-with-Glass-Lid-Sage-3950450/328831462', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:336 2024-09-18 23:36:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:337 2024-09-18 23:36:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Vigoro-Peace-Lily-Indoor-Plant-in-6-in-White-Ribbed-Plastic-Décor-Planter-Avg-Shipping-Height-1-2-ft-Tall-2-Pack-CO-SY06-3-VI-WH/318745853', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:338 2024-09-18 23:36:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:339 2024-09-18 23:36:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:340 2024-09-18 23:36:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:341 2024-09-18 23:36:49 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Norfolk-Island-Pine-Indoor-Plant-in-6-in-Ugly-Sweater-Décor-Pot-Avg-Shipping-Height-1-2-ft-Tall-6NORFOLKPINE2PK/306717715', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:50 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Grower-s-Choice-Sansevieria-Indoor-Snake-Plant-in-10-in-Gray-Décor-Pot-Avg-Shipping-Height-1-2-ft-Tall-10SANSLTAUPEMOD/307917433', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:50 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/BergHOFF-Balance-3-1qt-Nonstick-Recycled-Aluminum-Sauté-Pan-10-25-in-with-Glass-Lid-Moonmist-3950426/328831459', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:342 2024-09-18 23:36:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:343 2024-09-18 23:36:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:344 2024-09-18 23:36:54 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Yucca-Cane-Indoor-Plant-in-8-75-White-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-10YUCREAMOD/307885320', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:345 2024-09-18 23:36:55 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Grower-s-Choice-Sansevieria-Indoor-Snake-Plant-in-10-in-White-Décor-Pot-Avg-Shipping-Height-1-2-ft-Tall-10SANSLCREAMMOD/307930288', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:346 2024-09-18 23:36:57 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Solar-Outdoor-Lights-43-in-Premium-99-LEDs-Solar-Torch-Lights-for-Garden-Décor-2-Pack-B07DW468T1/326472158', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:36:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:347 2024-09-18 23:36:59 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Roebellini-Pygmy-Date-Palm-Indoor-Plant-in-10-in-Natural-Décor-Basket-Avg-Shipping-Height-3-4-ft-Tall-CO-PXR-3-SEA/319495564', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:348 2024-09-18 23:37:01 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/24-in-Peanuts-Led-3D-Prelit-Yard-Décor-Skating-Snoopy-PW-36246/325909963', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:349 2024-09-18 23:37:02 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/BergHOFF-Leo-3-2-qt-Nonstick-Cast-Aluminum-Sauté-Pan-10-25-in-Grey-2219097/328831463', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:350 2024-09-18 23:37:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:351 2024-09-18 23:37:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:352 2024-09-18 23:37:07 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Croton-Petra-Indoor-Plant-in-8-75-Cream-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-10CROPECREAMOD/308040698', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Fiddle-Leaf-Fig-Indoor-Plant-in-10-in-White-Décor-Pot-Avg-Shipping-Height-1-2-ft-Tall-10PANPAR2FT/316780300', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Monstera-Deliciosa-Swiss-Cheese-Indoor-Plant-in-9-25-in-Natural-Décor-Basket-Avg-Shipping-Height-2-3-ft-Tall-10PHILOMONSEA/309479571', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:353 2024-09-18 23:37:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Bonnie-Plants-19-3-oz-Habañero-Hot-Pepper-Plant-3100/300718715', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:354 2024-09-18 23:37:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:355 2024-09-18 23:37:14 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Calphalon-Classic-5-qt-Nonstick-Aluminum-12-in-All-Purpose-Sauté-Pan-985120075M/326358288', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:356 2024-09-18 23:37:14 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/13-in-x-18-5-in-Fiesta-Día-de-Muertos-Garden-Flag-Double-Sided-Falltime-Decorative-Vertical-Flags-HDG130405-BO/323291398', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ZUHNË-Modena-21-in-Stainless-Steel-Undermount-Kitchen-Sink-with-Accessories-16-Gauge-Single-Bowl-for-24-in-Cabinet-Modena21/314486147', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:357 2024-09-18 23:37:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:358 2024-09-18 23:37:18 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Rain-Forest-20-in-Dia-x-18-in-H-Antúrio-Rolled-Rim-Plastic-Planter-R-0302-050-045-02/326750858', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:359 2024-09-18 23:37:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:360 2024-09-18 23:37:21 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Dracaena-Indoor-Plant-in-10-in-White-Wood-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-CO-DMC10-CYLSTD/320662736', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Butterfly-Cat-Metal-Garden-Cutout-Wrought-Iron-Insert-Solar-Decoration-For-Outdoor-Décor-2-pack-GCWIIBC2P/328678226', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:361 2024-09-18 23:37:23 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Mass-Cane-Indoor-Plant-in-8-78-in-Gray-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-10MASSTAUPEMOD/307940448', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/2-Tier-7-08-in-D-×-15-35-in-W-Under-Sink-Organizer-Storage-Drawer-Basket-Organizer-with-Hooks-Hanging-Cup-ABS-SNSA22IN495/328886105', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:362 2024-09-18 23:37:25 [scrapy.extensions.logstats] INFO: Crawled 150 pages (at 0 pages/min), scraped 906362 items (at 87136 items/min) 2024-09-18 23:37:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:363 2024-09-18 23:37:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:364 2024-09-18 23:37:30 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ZUHNË-Prato-30-in-Single-Bowl-Farmhouse-Curved-Apron-Front-Stainless-Steel-Kitchen-Sink-16-Gauge-Prato30/314603635', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:365 2024-09-18 23:37:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:366 2024-09-18 23:37:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:367 2024-09-18 23:37:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:368 2024-09-18 23:37:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Costa-Farms-Sago-Palm-Indoor-Plant-in-10-in-White-Décor-Pot-Avg-Shipping-Height-2-3-ft-Tall-CO-RV10-3-CYL/319268626', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ZUHNË-Roma-Brushed-16-Gauge-Stainless-Steel-32-in-Double-Bowl-Undermount-Kitchen-Sink-with-Bottom-Grid-Roma32/315675108', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:369 2024-09-18 23:37:37 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/1-71-Pint-Organic-Jalapeño-Pepper-Plants-4-Pack-PEPJAL000OVQV/314113343', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:370 2024-09-18 23:37:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:371 2024-09-18 23:37:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:372 2024-09-18 23:37:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Vigoro-Zamioculcas-Zamiifolia-ZZ-Indoor-Plant-in-6-in-White-Ribbed-Plastic-Décor-Planter-Avg-Shipping-Height-1-2-ft-2-Pack-CO-ZZ-VI-WH/318745760', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:373 2024-09-18 23:37:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Rain-Forest-18-5-in-Dia-x-18-5-in-H-Milão-Antique-Camurca-Plastic-Planter-047-047-05/327863554', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:37:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:374 2024-09-18 23:37:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:375 2024-09-18 23:37:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:376 2024-09-18 23:37:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:377 2024-09-18 23:37:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:378 2024-09-18 23:37:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:379 2024-09-18 23:37:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:380 2024-09-18 23:37:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:381 2024-09-18 23:37:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:382 2024-09-18 23:38:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:383 2024-09-18 23:38:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:384 2024-09-18 23:38:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:385 2024-09-18 23:38:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:386 2024-09-18 23:38:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:387 2024-09-18 23:38:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:388 2024-09-18 23:38:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:389 2024-09-18 23:38:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:390 2024-09-18 23:38:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:391 2024-09-18 23:38:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:392 2024-09-18 23:38:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:393 2024-09-18 23:38:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:394 2024-09-18 23:38:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:395 2024-09-18 23:38:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:396 2024-09-18 23:38:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:397 2024-09-18 23:38:25 [scrapy.extensions.logstats] INFO: Crawled 150 pages (at 0 pages/min), scraped 994026 items (at 87664 items/min) 2024-09-18 23:38:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:398 2024-09-18 23:38:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:399 2024-09-18 23:38:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:400 2024-09-18 23:38:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:401 2024-09-18 23:38:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:402 2024-09-18 23:38:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:403 2024-09-18 23:38:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:404 2024-09-18 23:38:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:405 2024-09-18 23:38:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:406 2024-09-18 23:38:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:407 2024-09-18 23:38:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:408 2024-09-18 23:38:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:409 2024-09-18 23:38:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:410 2024-09-18 23:38:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:411 2024-09-18 23:38:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:412 2024-09-18 23:38:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:413 2024-09-18 23:38:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:414 2024-09-18 23:38:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:415 2024-09-18 23:38:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:416 2024-09-18 23:39:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:417 2024-09-18 23:39:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:418 2024-09-18 23:39:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:419 2024-09-18 23:39:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:420 2024-09-18 23:39:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:421 2024-09-18 23:39:14 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/22-05-in-W×22-05-in-D×39-57-in-Gray-Bath-Vanity-Cabinet-with-1-White-Sink-White-Marble-Countertop-Fully-Assembled-HDSA17IN215/328304188', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:422 2024-09-18 23:39:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:423 2024-09-18 23:39:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:424 2024-09-18 23:39:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:425 2024-09-18 23:39:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:426 2024-09-18 23:39:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:427 2024-09-18 23:39:25 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/DEERPORT-DÉCOR-2-Pack-of-16-ft-32-ft-total-80-LED-Blue-Mini-Rope-Light-ML11-MRL16-BL-2/301619421', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:428 2024-09-18 23:39:25 [scrapy.extensions.logstats] INFO: Crawled 194 pages (at 44 pages/min), scraped 1070724 items (at 76698 items/min) 2024-09-18 23:39:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/19-in-Brown-Contemporary-Metal-Winding-Ivy-Table-Desk-Lamp-with-Fabric-Shade-for-Home-Décor-CWT-2006-BW/325853256', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:429 2024-09-18 23:39:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:430 2024-09-18 23:39:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:431 2024-09-18 23:39:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:432 2024-09-18 23:39:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:433 2024-09-18 23:39:35 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Outdoor-Throw-Pillow-Covers-Dark-Grey-Cozy-Soft-Velvet-Square-Decorative-Pillow-Cases-for-Farmhouse-Home-Décor-2-Pack-B07Y7X5MDK/326643504', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:434 2024-09-18 23:39:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:435 2024-09-18 23:39:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:436 2024-09-18 23:39:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:437 2024-09-18 23:39:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-36-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-803836-FN-LW-NM/325922968', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:438 2024-09-18 23:39:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:439 2024-09-18 23:39:45 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/22-2-in-W-×-36-37-in-D-×-36-57-in-H-White-Bath-Vanity-Cabinet-with-1-Sink-Marble-Countertop-Fully-Assembled-HDSA17IN216/328304189', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:440 2024-09-18 23:39:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:441 2024-09-18 23:39:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:442 2024-09-18 23:39:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:443 2024-09-18 23:39:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-48-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-and-Mirror-803848-FW-RG/325922894', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:444 2024-09-18 23:39:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:445 2024-09-18 23:39:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:446 2024-09-18 23:39:58 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-72-in-W-x-22-in-D-x-34-in-H-Free-standing-Double-Bathroom-Vanity-in-Fir-Wood-Grey-with-White-Composite-Top-804172-FY-LW-NM/326214394', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:59 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Glacier-Bay-Faucet-Ceramic-Cartridge-Assembly-External-Diameter-25-mm-Ø8-8-mm-RP90115/301158148', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:39:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:447 2024-09-18 23:40:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:448 2024-09-18 23:40:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:449 2024-09-18 23:40:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:450 2024-09-18 23:40:04 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-72-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-803872-FW-RG-NM/325922978', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/9-ft-×-9-ft-Square-Cantilever-Umbrella-For-Your-Outdoor-Space-240-g-Solution-Dyed-Fabric-Aluminum-Frame-in-Gray-J-99GY/329056123', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:451 2024-09-18 23:40:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:452 2024-09-18 23:40:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/17-25-in-Pink-Contemporary-Ceramic-Stacking-Stones-Table-Desk-Lamp-for-Home-Décor-CWT-2014-PN/325853259', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:08 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-84-in-W-x-22-in-D-x-34-in-H-Double-Bathroom-Vanity-in-Fir-Wood-Grey-with-White-Composite-top-and-Mirror-804184-FY-LW/326214373', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:453 2024-09-18 23:40:10 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-84-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-and-Mirror-803884-FW-RG/325922903', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:454 2024-09-18 23:40:12 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-72-in-W-x-22-in-D-x-34-in-H-Free-standing-Double-Bathroom-Vanity-in-Fir-Wood-White-with-Gray-Composite-Top-804172-FW-RG-NM/326214393', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:455 2024-09-18 23:40:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:456 2024-09-18 23:40:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/3-ft-L-×-1-ft-W-Black-Grow-Tent-for-Indoor-Seedling-Plant-Growing-Water-Resistant-GLTENTXS3/323634460', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:457 2024-09-18 23:40:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:458 2024-09-18 23:40:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:459 2024-09-18 23:40:21 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-24-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-and-Mirror-803824-FN-LW/325922886', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:460 2024-09-18 23:40:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:461 2024-09-18 23:40:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:462 2024-09-18 23:40:25 [scrapy.extensions.logstats] INFO: Crawled 194 pages (at 0 pages/min), scraped 1156342 items (at 85618 items/min) 2024-09-18 23:40:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-36-in-W-x-22-in-D-x-34-in-H-Free-standing-Single-Bathroom-Vanity-in-Fir-Wood-Black-with-White-Composite-Top-804136-FB-LW-NM/326214383', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:463 2024-09-18 23:40:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:464 2024-09-18 23:40:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-24-in-W-x-22-in-D-x-34-in-H-Free-standing-Single-Bathroom-Vanity-in-Fir-Wood-Black-with-Composite-top-804124-FB-LW-NM/326214380', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:465 2024-09-18 23:40:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:466 2024-09-18 23:40:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:467 2024-09-18 23:40:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-72-in-W-x-22-in-D-x-34-in-H-Double-Bathroom-Vanity-in-Fir-Wood-Black-with-White-Composite-top-and-Mirror-804172-FB-LW/326214368', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:468 2024-09-18 23:40:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-24-in-W-x-22-in-D-x-34-in-H-Free-standing-Single-Bathroom-Vanity-in-Fir-Wood-Grey-with-Composite-top-804124-FY-LW-NM/326214382', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:469 2024-09-18 23:40:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:470 2024-09-18 23:40:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:471 2024-09-18 23:40:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:472 2024-09-18 23:40:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-60-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-and-Mirror-803860M-FN-LW/325922896', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Glacier-Bay-Faucet-Ceramic-Cartridge-Assembly-External-Diameter-35-mm-Ø4-8-mm-RP90118/301158214', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:473 2024-09-18 23:40:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-84-in-W-x-22-in-D-x-34-in-H-Double-Bathroom-Vanity-in-Fir-Wood-Black-with-White-Composite-top-and-Mirror-804184-FB-LW/326214371', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:474 2024-09-18 23:40:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:475 2024-09-18 23:40:49 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-72-in-W-x-22-in-D-x-34-in-H-Double-Bathroom-Vanity-in-Fir-Wood-White-with-Gray-Composite-top-and-Mirror-804172-FW-RG/326214369', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:476 2024-09-18 23:40:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:477 2024-09-18 23:40:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:478 2024-09-18 23:40:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:479 2024-09-18 23:40:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:480 2024-09-18 23:40:58 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-60-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Grey-with-White-Composite-Stone-Top-and-Mirror-803860-FY-LW/326051171', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:40:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:481 2024-09-18 23:41:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:482 2024-09-18 23:41:00 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Glacier-Bay-Faucet-Ceramic-Cartridge-Assembly-External-Diameter-25-mm-Ø7-6-mm-RP90003/206116616', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:483 2024-09-18 23:41:03 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-36-in-W-x-22-in-D-x-34-in-H-Free-standing-Single-Bathroom-Vanity-in-Fir-Wood-White-withg-Gray-Composite-Top-804136-FW-RG-NM/326214384', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:484 2024-09-18 23:41:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-60-in-W-x-22-in-D-x-34-in-H-Double-Bathroom-Vanity-in-Fir-Wood-Grey-with-White-Composite-top-and-Mirror-804160M-FY-LW/326214367', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:485 2024-09-18 23:41:06 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-84-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-803884-FN-LW-NM/325922980', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:06 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/14-96-in-W×7-28-in-D×20-28-in-H-9-Drawers-Gray-Bath-Vanity-Cabinet-with-1-Sink-Marble-Standard-Top-Fully-Assembled-HDSA17IN213/328304176', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:06 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-24-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Grey-with-White-Composite-Stone-Top-803824-FY-LW-NM/325922967', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:486 2024-09-18 23:41:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:487 2024-09-18 23:41:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/9-ft-×-9-ft-Outdoor-Square-Cantilever-LED-Patio-Umbrella-240-g-Solution-Dyed-Fabric-Aluminum-Frame-in-Taupe-J-99TP-LED/329056208', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:488 2024-09-18 23:41:11 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-21-7-in-W-x-15-7-in-D-x-71-9-in-H-Floor-Black-Linen-Cabinet-for-Bathroom-Kitchen-and-Living-Room-804122-SCA-FB/326324406', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:11 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-48-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-803848-FN-LW-NM/325922971', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:489 2024-09-18 23:41:13 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-21-7-in-W-x-15-7-in-D-x-71-9-in-H-Floor-Gray-Linen-Cabinet-for-Bathroom-and-Living-Room-804122-SCA-FY/326324409', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:490 2024-09-18 23:41:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:491 2024-09-18 23:41:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-60-in-W-x-22-in-D-x-34-in-H-Free-standing-Double-Bathroom-Vanity-in-Fir-Wood-Black-with-White-Composite-Top-804160M-FB-LWN/326214389', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:492 2024-09-18 23:41:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:493 2024-09-18 23:41:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:494 2024-09-18 23:41:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-48-in-W-x-22-in-D-x-34-in-H-Single-Bathroom-Vanity-in-Fir-Wood-Grey-with-White-Composite-top-and-Mirror-804148-FY-LW/326214364', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:495 2024-09-18 23:41:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-36-in-W-x-22-in-D-x-34-in-H-Single-Bathroom-Vanity-in-Fir-Wood-Grey-with-White-Composite-top-and-Mirror-804136-FY-LW/326214361', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:496 2024-09-18 23:41:25 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-24-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Grey-with-White-Composite-Stone-Top-and-Mirror-803824-FY-LW/325922888', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:25 [scrapy.extensions.logstats] INFO: Crawled 194 pages (at 0 pages/min), scraped 1243047 items (at 86705 items/min) 2024-09-18 23:41:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:497 2024-09-18 23:41:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:498 2024-09-18 23:41:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:499 2024-09-18 23:41:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:500 2024-09-18 23:41:31 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-60-in-W-x-22-in-D-x-34-in-H-Free-standing-Double-Bathroom-Vanity-in-Fir-Wood-White-with-Gray-Composite-Top-804160M-FW-RGN/326214390', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:32 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-60-in-W-x-22-in-D-x-34-in-H-Double-Bathroom-Vanity-in-Fir-Wood-Black-with-White-Composite-top-and-Mirror-804160M-FB-LW/326214365', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:501 2024-09-18 23:41:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:502 2024-09-18 23:41:35 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-72-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-Grey-with-White-Composite-Stone-Top-and-Mirror-803872-FY-LW/325922901', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:503 2024-09-18 23:41:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-21-7-in-W-x-15-7-in-D-x-72-in-H-Brown-Storage-Linen-Cabinet-for-Bathroom-Kitchen-and-Living-Room-803822-SCA-FN/326165303', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:504 2024-09-18 23:41:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-21-7-in-W-x-15-7-in-D-x-71-9-in-H-Floor-White-Linen-Cabinet-for-Bathroom-Kitchen-and-Living-Room-804122-SCA-FW/326324408', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:505 2024-09-18 23:41:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:506 2024-09-18 23:41:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/14-7-in-Gray-Contemporary-Ceramic-Stacking-Stones-Table-Desk-Lamp-for-Home-Décor-CWT-2015-GY/325853260', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:507 2024-09-18 23:41:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:508 2024-09-18 23:41:45 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/27-60-in-W-×-63-80-in-H-×-7-70-in-D-White-Over-The-Toilet-Bathroom-Cabinet-Adjustable-Shelf-Collect-Cabinet-RS-27OTTS2-WB/324147419', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:46 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/14-96-in-W-×-7-28-in-D-×-20-28-in-H-9-Drawers-White-Bath-Vanity-Cabinet-1-Sink-Marble-Countertop-Fully-Assembled-HDSA17IN214/328304187', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:509 2024-09-18 23:41:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:510 2024-09-18 23:41:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:511 2024-09-18 23:41:51 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-24-in-W-x-22-in-D-x-34-in-H-Single-Bathroom-Vanity-in-Fir-Wood-Grey-with-Composite-top-and-Mirror-804124-FY-LW/326214358', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:512 2024-09-18 23:41:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-36-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Grey-with-White-Composite-Stone-Top-803836-FY-LW-NM/325922970', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-24-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-and-Mirror-803824-FW-RG/325922887', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:53 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-60-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-Grey-with-White-Composite-Stone-Top-803860M-FY-LWN/325922976', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:513 2024-09-18 23:41:54 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-48-in-W-x-22-in-D-x-34-in-H-Free-Standing-Single-Sink-Bathroom-Vanity-in-Fir-Wood-Grey-with-White-Composite-Top-804148-FY-LW-NM/326214388', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:514 2024-09-18 23:41:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:515 2024-09-18 23:41:57 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-60-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-and-Mirror-803860-FW-RG/326051170', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:57 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/VOLßN-5-Light-Aged-Brass-Linear-Pendant-6617-5-80/326047544', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:516 2024-09-18 23:41:59 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-60-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-803860-FW-RG-NM/326051117', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:59 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-36-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-803836-FW-RG-NM/325922969', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:41:59 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-24-in-W-x-22-in-D-x-34-in-H-Single-Bathroom-Vanity-in-Fir-Wood-Black-with-Composite-top-and-Mirror-804124-FB-LW/326214356', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:517 2024-09-18 23:42:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:518 2024-09-18 23:42:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:519 2024-09-18 23:42:04 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/14-96-in-W-×-7-28-in-D-×-20-28-in-H-5-Drawers-White-Bath-Vanity-Cabinet-with-1-Sink-Marble-Countertop-Fully-Assembled-HDSA17IN212/328304175', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:520 2024-09-18 23:42:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-48-in-W-x-22-in-D-x-34-in-H-Single-Bathroom-Vanity-in-Fir-Wood-White-with-Gray-Composite-top-and-Mirror-804148-FW-RG/326214363', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-60-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-803860M-FW-RGN/325922975', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:521 2024-09-18 23:42:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:522 2024-09-18 23:42:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-36-in-W-x-22-in-D-x-34-in-H-Single-Bathroom-Vanity-in-Fir-Wood-White-with-Gray-Composite-top-and-Mirror-804136-FW-RG/326214360', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:523 2024-09-18 23:42:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:524 2024-09-18 23:42:15 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/9-ft-×-9-ft-Outdoor-Square-Cantilever-LED-Patio-Umbrella-240-g-Solution-Dyed-Fabric-Aluminum-Frame-in-Anthracite-J-99LY-LED/329056210', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:525 2024-09-18 23:42:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:526 2024-09-18 23:42:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:527 2024-09-18 23:42:17 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-21-7-in-W-x-15-7-in-D-x-72-in-H-Floor-Gray-Linen-Cabinet-803822-SCA-FY/326165305', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:528 2024-09-18 23:42:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:529 2024-09-18 23:42:22 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-72-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-and-Mirror-803872-FN-LW/325922899', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:530 2024-09-18 23:42:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-60-in-W-x-22-in-D-x-34-in-H-Double-Bathroom-Vanity-in-Fir-Wood-White-with-Gray-Composite-top-and-Mirror-804160M-FW-RG/326214366', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-48-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-Grey-with-White-Composite-Stone-Top-803848-FY-LW-NM/325922973', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Volßn-1-Light-Aged-Brass-Pendant-3317-80/326047608', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-36-in-W-x-22-in-D-x-34-in-H-Single-Sink-Bath-Vanity-in-Fir-Wood-White-with-Grey-Composite-Stone-Top-and-Mirror-803836-FW-RG/325922891', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:531 2024-09-18 23:42:25 [scrapy.extensions.logstats] INFO: Crawled 194 pages (at 0 pages/min), scraped 1329018 items (at 85971 items/min) 2024-09-18 23:42:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:532 2024-09-18 23:42:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:533 2024-09-18 23:42:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-24-in-W-x-22-in-D-x-34-in-H-Free-standing-Single-Bathroom-Vanity-in-Fir-Wood-White-with-Composite-top-804124-FW-RG-NM/326214381', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/52-in-W-×-54-in-D-×-70-9-in-H-Iron-Hammock-Stand-in-Black-SYXY91294037/324052936', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:29 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-84-in-W-x-22-in-D-x-34-in-H-Free-standing-Double-Bathroom-Vanity-in-Fir-Wood-Grey-with-White-Composite-Top-804184-FY-LW-NM/326214397', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:534 2024-09-18 23:42:30 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-84-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-and-Mirror-803884-FN-LW/325922902', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:535 2024-09-18 23:42:33 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/9-ft-×-9-ft-Outdoor-Square-Cantilever-LED-Patio-Umbrella-240-g-Solution-Dyed-Fabric-Aluminum-Frame-in-Navy-Blue-J-99BL-LED/329056207', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:33 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-León-60-in-W-x-22-in-D-x-34-in-H-Double-Sink-Bath-Vanity-in-Fir-Wood-Brown-with-White-Composite-Stone-Top-803860M-FN-LWN/325922974', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:536 2024-09-18 23:42:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:537 2024-09-18 23:42:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/ROSWELL-Cádiz-48-in-W-x-22-in-D-x-34-in-H-Single-Bathroom-Vanity-in-Fir-Wood-Black-with-White-Composite-top-and-Mirror-804148-FB-LW/326214362', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:538 2024-09-18 23:42:36 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/22-in-W-x-36-in-D-×-35-5-in-H-White-Marble-Free-Standing-Single-Bathroom-Vanity-with-Engineered-Marble-Top-HDSA17BA075/328871059', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:42:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:539 2024-09-18 23:42:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:540 2024-09-18 23:42:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:541 2024-09-18 23:42:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:542 2024-09-18 23:42:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:543 2024-09-18 23:42:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:544 2024-09-18 23:42:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:545 2024-09-18 23:42:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:546 2024-09-18 23:43:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:547 2024-09-18 23:43:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:548 2024-09-18 23:43:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:549 2024-09-18 23:43:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:550 2024-09-18 23:43:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:551 2024-09-18 23:43:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:552 2024-09-18 23:43:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:553 2024-09-18 23:43:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:554 2024-09-18 23:43:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:555 2024-09-18 23:43:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:556 2024-09-18 23:43:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:557 2024-09-18 23:43:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:558 2024-09-18 23:43:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:559 2024-09-18 23:43:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:560 2024-09-18 23:43:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:561 2024-09-18 23:43:25 [scrapy.extensions.logstats] INFO: Crawled 236 pages (at 42 pages/min), scraped 1404591 items (at 75573 items/min) 2024-09-18 23:43:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:562 2024-09-18 23:43:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:563 2024-09-18 23:43:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:564 2024-09-18 23:43:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:565 2024-09-18 23:43:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:566 2024-09-18 23:43:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:567 2024-09-18 23:43:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:568 2024-09-18 23:43:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:569 2024-09-18 23:43:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:570 2024-09-18 23:43:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:571 2024-09-18 23:43:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:572 2024-09-18 23:43:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:573 2024-09-18 23:43:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:574 2024-09-18 23:43:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:575 2024-09-18 23:43:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:576 2024-09-18 23:43:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:577 2024-09-18 23:43:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:578 2024-09-18 23:43:48 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/HDX-13-Gal-Â-FLEXÂ-White-Drawstring-Kitchen-Trash-Bags-55-Count-HDX959537/203472837', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:43:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:579 2024-09-18 23:43:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:580 2024-09-18 23:43:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:581 2024-09-18 23:43:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:582 2024-09-18 23:43:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:583 2024-09-18 23:43:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:584 2024-09-18 23:43:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:585 2024-09-18 23:43:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:586 2024-09-18 23:43:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:587 2024-09-18 23:44:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:588 2024-09-18 23:44:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:589 2024-09-18 23:44:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:590 2024-09-18 23:44:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:591 2024-09-18 23:44:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/BEHR-PREMIUM-PLUS-8-oz-MQ2-33-Parisian-Café-Semi-Gloss-Interior-Exterior-Paint-Primer-Color-Sample-B330416/326267315', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:44:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:592 2024-09-18 23:44:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:593 2024-09-18 23:44:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:594 2024-09-18 23:44:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:595 2024-09-18 23:44:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:596 2024-09-18 23:44:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:597 2024-09-18 23:44:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:598 2024-09-18 23:44:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:599 2024-09-18 23:44:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:600 2024-09-18 23:44:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:601 2024-09-18 23:44:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:602 2024-09-18 23:44:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:603 2024-09-18 23:44:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:604 2024-09-18 23:44:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:605 2024-09-18 23:44:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:606 2024-09-18 23:44:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:607 2024-09-18 23:44:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:608 2024-09-18 23:44:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:609 2024-09-18 23:44:25 [scrapy.extensions.logstats] INFO: Crawled 236 pages (at 0 pages/min), scraped 1523780 items (at 119189 items/min) 2024-09-18 23:44:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:610 2024-09-18 23:44:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:611 2024-09-18 23:44:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:612 2024-09-18 23:44:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:613 2024-09-18 23:44:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:614 2024-09-18 23:44:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:615 2024-09-18 23:44:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:616 2024-09-18 23:44:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:617 2024-09-18 23:44:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:618 2024-09-18 23:44:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:619 2024-09-18 23:44:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:620 2024-09-18 23:44:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:621 2024-09-18 23:44:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:622 2024-09-18 23:44:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:623 2024-09-18 23:44:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:624 2024-09-18 23:44:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:625 2024-09-18 23:44:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:626 2024-09-18 23:44:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:627 2024-09-18 23:44:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:628 2024-09-18 23:44:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:629 2024-09-18 23:44:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:630 2024-09-18 23:44:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:631 2024-09-18 23:44:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:632 2024-09-18 23:44:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:633 2024-09-18 23:44:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:634 2024-09-18 23:44:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:635 2024-09-18 23:44:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:636 2024-09-18 23:44:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:637 2024-09-18 23:44:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:638 2024-09-18 23:45:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:639 2024-09-18 23:45:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:640 2024-09-18 23:45:02 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/BEHR-PREMIUM-1-qt-MQ2-33-Parisian-Café-Semi-Gloss-Enamel-Interior-Exterior-Cabinet-Door-Trim-Paint-712004/328091599', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:45:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:641 2024-09-18 23:45:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:642 2024-09-18 23:45:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:643 2024-09-18 23:45:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:644 2024-09-18 23:45:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:645 2024-09-18 23:45:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:646 2024-09-18 23:45:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:647 2024-09-18 23:45:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:648 2024-09-18 23:45:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:649 2024-09-18 23:45:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:650 2024-09-18 23:45:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:651 2024-09-18 23:45:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:652 2024-09-18 23:45:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:653 2024-09-18 23:45:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:654 2024-09-18 23:45:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:655 2024-09-18 23:45:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:656 2024-09-18 23:45:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:657 2024-09-18 23:45:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:658 2024-09-18 23:45:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:659 2024-09-18 23:45:25 [scrapy.extensions.logstats] INFO: Crawled 236 pages (at 0 pages/min), scraped 1649889 items (at 126109 items/min) 2024-09-18 23:45:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:660 2024-09-18 23:45:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:661 2024-09-18 23:45:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:662 2024-09-18 23:45:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:663 2024-09-18 23:45:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:664 2024-09-18 23:45:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:665 2024-09-18 23:45:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:666 2024-09-18 23:45:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:667 2024-09-18 23:45:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:668 2024-09-18 23:45:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:669 2024-09-18 23:45:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:670 2024-09-18 23:45:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:671 2024-09-18 23:45:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:672 2024-09-18 23:45:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:673 2024-09-18 23:45:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:674 2024-09-18 23:45:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:675 2024-09-18 23:45:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:676 2024-09-18 23:45:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:677 2024-09-18 23:45:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:678 2024-09-18 23:45:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:679 2024-09-18 23:45:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:680 2024-09-18 23:45:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:681 2024-09-18 23:45:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:682 2024-09-18 23:45:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:683 2024-09-18 23:45:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:684 2024-09-18 23:45:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:685 2024-09-18 23:45:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:686 2024-09-18 23:45:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:687 2024-09-18 23:45:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:688 2024-09-18 23:46:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:689 2024-09-18 23:46:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:690 2024-09-18 23:46:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:691 2024-09-18 23:46:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:692 2024-09-18 23:46:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:693 2024-09-18 23:46:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:694 2024-09-18 23:46:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:695 2024-09-18 23:46:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:696 2024-09-18 23:46:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:697 2024-09-18 23:46:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:698 2024-09-18 23:46:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:699 2024-09-18 23:46:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:700 2024-09-18 23:46:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:701 2024-09-18 23:46:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:702 2024-09-18 23:46:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:703 2024-09-18 23:46:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:704 2024-09-18 23:46:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:705 2024-09-18 23:46:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:706 2024-09-18 23:46:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:707 2024-09-18 23:46:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:708 2024-09-18 23:46:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:709 2024-09-18 23:46:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:710 2024-09-18 23:46:25 [scrapy.extensions.logstats] INFO: Crawled 236 pages (at 0 pages/min), scraped 1777324 items (at 127435 items/min) 2024-09-18 23:46:26 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:711 2024-09-18 23:46:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:712 2024-09-18 23:46:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:713 2024-09-18 23:46:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:714 2024-09-18 23:46:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:715 2024-09-18 23:46:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:716 2024-09-18 23:46:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:717 2024-09-18 23:46:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:718 2024-09-18 23:46:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:719 2024-09-18 23:46:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:720 2024-09-18 23:46:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:721 2024-09-18 23:46:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:722 2024-09-18 23:46:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:723 2024-09-18 23:46:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:724 2024-09-18 23:46:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:725 2024-09-18 23:46:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:726 2024-09-18 23:46:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:727 2024-09-18 23:46:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:728 2024-09-18 23:46:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:729 2024-09-18 23:46:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:730 2024-09-18 23:46:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:731 2024-09-18 23:46:51 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:732 2024-09-18 23:46:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:733 2024-09-18 23:46:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:734 2024-09-18 23:46:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:735 2024-09-18 23:46:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:736 2024-09-18 23:46:56 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:737 2024-09-18 23:46:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:738 2024-09-18 23:46:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:739 2024-09-18 23:47:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:740 2024-09-18 23:47:02 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:741 2024-09-18 23:47:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:742 2024-09-18 23:47:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:743 2024-09-18 23:47:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:744 2024-09-18 23:47:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:745 2024-09-18 23:47:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:746 2024-09-18 23:47:09 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:747 2024-09-18 23:47:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:748 2024-09-18 23:47:11 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:749 2024-09-18 23:47:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:750 2024-09-18 23:47:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:751 2024-09-18 23:47:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:752 2024-09-18 23:47:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:753 2024-09-18 23:47:17 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:754 2024-09-18 23:47:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:755 2024-09-18 23:47:20 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:756 2024-09-18 23:47:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:757 2024-09-18 23:47:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:758 2024-09-18 23:47:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:759 2024-09-18 23:47:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:760 2024-09-18 23:47:25 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.homedepot.com/sitemap/P/PIPs/22/022-004-0.xml>: HTTP status code is not handled or not allowed 2024-09-18 23:47:25 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.homedepot.com/sitemap/P/PIPs/24/024-004-0.xml>: HTTP status code is not handled or not allowed 2024-09-18 23:47:25 [scrapy.extensions.logstats] INFO: Crawled 238 pages (at 2 pages/min), scraped 1901693 items (at 124369 items/min) 2024-09-18 23:47:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:761 2024-09-18 23:47:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Coral-090481/315616467', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:28 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Calabaza-Orange-015667/313320595', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:29 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Emac-Novopeldaño-Mini-Matt-Silver-1-4-in-x-8-1-2-in-Aluminum-Tile-Edging-trim-NPMN1314/324337808', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:762 2024-09-18 23:47:29 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Sunbright-Yellow-026811/313322836', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:763 2024-09-18 23:47:31 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Sapphire-Blue-018741/313320611', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:31 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Chile-Pepper-026815/313320695', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:32 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:764 2024-09-18 23:47:32 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/EMAC-Novopeldaño-Nori-Cinnamon-1-9-16-in-x-8-1-2-in-ASTRA-Tile-Edging-trim-NPASN1631/324898908', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:32 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Golden-Yellow-090479/315616471', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:765 2024-09-18 23:47:33 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Sangria-082331/313320812', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:766 2024-09-18 23:47:34 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Sunbright-Yellow-013815/313322901', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:35 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Aliénor-Machine-Washable-Blue-8-ft-x-10-ft-Tribal-Area-Rug-GRHN04A-8010/326900753', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:767 2024-09-18 23:47:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:768 2024-09-18 23:47:37 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Pink-090480/315616469', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:38 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:769 2024-09-18 23:47:38 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Calabaza-Orange-026813/313322843', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:39 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Sangria-026818/313320702', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:770 2024-09-18 23:47:40 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Tangerine-026812/313320755', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:771 2024-09-18 23:47:40 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Burro-Brown-026823/313322871', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:40 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Passion-Purple-026829/313323486', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:41 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Santa-Fe-Red-026814/313322847', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:41 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Órla-Tribal-Bordered-Machine-Washable-Rust-8-ft-x-10-ft-Area-Rug-BIRV97A-8010/326678412', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:41 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Passion-Purple-013819/313320608', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:772 2024-09-18 23:47:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Mantilla-Black-013821/313320643', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Lifeproof-with-Petproof-Technology-Gazelle-II-Crème-Brulee-Beige-55-oz-Triexta-Texture-Installed-Carpet-0640D-27-12/300387441', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Shadow-Grey-082335/313320636', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:773 2024-09-18 23:47:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Burro-Brown-082332/313320628', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Merola-Tile-Manises-Décor-Cuero-13-1-8-in-x-13-1-8-in-Ceramic-Floor-and-Wall-Take-Home-Tile-Sample-S1FEM13MNCD/326752905', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Chile-Pepper-066366/313320770', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Lime-Green-066369/313320619', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Tangerine-066365/313320881', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:774 2024-09-18 23:47:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Baja-Blue-066368/313320615', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:43 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Santa-Fe-Red-013818/313320598', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Havana-Brown-026825/313322875', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Baja-Blue-026820/313322859', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Shadow-Grey-026826/313322878', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Teal-090483/315616470', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:44 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Blanco-Blanco-026827/313320688', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:775 2024-09-18 23:47:45 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Rainforest-Green-013820/313320624', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:45 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-1-2-oz-Blue-Violet-090482/315616468', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:46 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/JACQUARD-Piñata-Alcohol-Ink-4-oz-Havana-Brown-082334/313320631', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:46 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:776 2024-09-18 23:47:47 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:777 2024-09-18 23:47:48 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:778 2024-09-18 23:47:49 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:779 2024-09-18 23:47:50 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/EMAC-Novopeldaño-Nori-Ash-1-9-16-in-x-8-1-2-in-ASTRA-Tile-Edging-trim-NPASN1701/324899716', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:50 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:780 2024-09-18 23:47:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Aliénor-Machine-Washable-Blue-4-ft-x-6-ft-Tribal-Area-Rug-GRHN04A-406/326900729', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:52 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:781 2024-09-18 23:47:52 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Merola-Tile-Chalk-Décor-Topos-3-in-x-11-3-4-in-Ceramic-Wall-Take-Home-Tile-Sample-S1WBT12CTO/323644241', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:47:53 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:782 2024-09-18 23:47:54 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:783 2024-09-18 23:47:55 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:784 2024-09-18 23:47:57 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:785 2024-09-18 23:47:58 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:786 2024-09-18 23:47:59 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:787 2024-09-18 23:47:59 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Órla-Tribal-Bordered-Machine-Washable-Rust-5-ft-x-8-ft-Area-Rug-BIRV97A-508/326679326', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:00 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:788 2024-09-18 23:48:00 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Úna-Lattice-Stain-Resistant-Machine-Washable-Beige-4-ft-x-6-ft-Area-Rug-HJTH02A-406/326587257', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:01 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:789 2024-09-18 23:48:03 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:790 2024-09-18 23:48:04 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Úna-Lattice-Stain-Resistant-Machine-Washable-Beige-9-ft-x-12-ft-Area-Rug-HJTH02A-9012/326587186', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:04 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:791 2024-09-18 23:48:05 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:792 2024-09-18 23:48:05 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Anaïs-Diamond-High-Low-Tassel-Light-Gray-8-ft-x-10-ft-Modern-Area-Rug-OZNN11A-8010/327350477', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:06 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:793 2024-09-18 23:48:07 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:794 2024-09-18 23:48:08 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:795 2024-09-18 23:48:09 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Renée-Machine-Washable-Multicolor-5-ft-x-8-ft-Tribal-Area-Rug-GRHN08A-508/326678799', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:10 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:796 2024-09-18 23:48:12 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:797 2024-09-18 23:48:13 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:798 2024-09-18 23:48:14 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:799 2024-09-18 23:48:15 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:800 2024-09-18 23:48:16 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:801 2024-09-18 23:48:18 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:802 2024-09-18 23:48:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:803 2024-09-18 23:48:19 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:804 2024-09-18 23:48:20 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Aliénor-Machine-Washable-Blue-5-ft-x-8-ft-Tribal-Area-Rug-GRHN04A-508/326900683', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:20 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Rugs-America-Rugs-America-Café-Royal-2-X-8ft-Indoor-Area-Rug-RA31114/325258971', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:21 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:805 2024-09-18 23:48:22 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:806 2024-09-18 23:48:23 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:807 2024-09-18 23:48:24 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/Rugs-America-Rugs-America-Café-Royal-2-X-4ft-Indoor-Area-Rug-RA31113/325258969', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:24 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:808 2024-09-18 23:48:25 [scrapy.extensions.logstats] INFO: Crawled 257 pages (at 19 pages/min), scraped 2022704 items (at 121011 items/min) 2024-09-18 23:48:25 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:809 2024-09-18 23:48:26 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Úna-Lattice-Stain-Resistant-Machine-Washable-Beige-5-ft-x-8-ft-Area-Rug-HJTH02A-508/326587180', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:27 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:810 2024-09-18 23:48:28 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:811 2024-09-18 23:48:29 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:812 2024-09-18 23:48:29 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Véronique-Transitional-Vines-Wool-Navy-5-ft-x-8-ft-Modern-Area-Rug-NECR05A-508/326900746', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:30 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:813 2024-09-18 23:48:31 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:814 2024-09-18 23:48:33 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:815 2024-09-18 23:48:34 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:816 2024-09-18 23:48:35 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:817 2024-09-18 23:48:36 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:818 2024-09-18 23:48:37 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:819 2024-09-18 23:48:39 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:820 2024-09-18 23:48:40 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:821 2024-09-18 23:48:41 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:822 2024-09-18 23:48:42 [scrapy.core.scraper] WARNING: Dropped: Validation failed! {'url': 'https://www.homedepot.com/p/nuLOOM-Joëlle-Light-Gray-8-ft-x-10-ft-Geometric-Tasseled-Area-Rug-RZET02A-8010/326900725', 'sku': None, 'name': None, 'supplier': 'HOME_DEPOT'} 2024-09-18 23:48:42 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:823 2024-09-18 23:48:43 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:824 2024-09-18 23:48:44 [scrapy.core.engine] INFO: Closing spider (finished) 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] ------------------------------ MONITORS ------------------------------ 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Extracted Items Monitor/test_stat_monitor... FAIL 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... FAIL 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Error Count Monitor/test_stat_monitor... SKIPPED (Unable to find 'log_count/ERROR' in job stats.) 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Warning Count Monitor/test_stat_monitor... FAIL 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Unwanted HTTP codes monitor/Should not hit the limit of unwanted http status... OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Field Coverage Monitor/test_check_if_field_coverage_rules_are_met... OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Retry Count monitor/Should not hit the limit of requests that reached the maximum retry amount... OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Downloader Exceptions monitor/test_stat_monitor... SKIPPED (Unable to find 'downloader/exception_count' in job stats.) 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Successful Requests monitor/Should have at least the minimum number of successful requests... OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] Total Requests monitor/Should not hit the total limit of requests... OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-09-18 23:48:44 [home_depot] ERROR: [Spidermon] ====================================================================== FAIL: Extracted Items Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'item_scraped_count' to be '>=' to '2181000.0'. Current value: '2062536' 2024-09-18 23:48:44 [home_depot] ERROR: [Spidermon] ====================================================================== FAIL: Item Validation Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'spidermon/validation/fields/errors' to be '<=' to '0.0'. Current value: '352' 2024-09-18 23:48:44 [home_depot] ERROR: [Spidermon] ====================================================================== FAIL: Warning Count Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'log_count/WARNING' to be '<=' to '1.0'. Current value: '352' 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] 11 monitors in 0.004s 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] FAILED (failures=3, skipped=2) 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] -------------------------- FINISHED ACTIONS -------------------------- 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] 0 actions in 0.000s 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] --------------------------- PASSED ACTIONS --------------------------- 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] 0 actions in 0.000s 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] OK 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] --------------------------- FAILED ACTIONS --------------------------- 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] ---------------------------------------------------------------------- 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] 0 actions in 0.000s 2024-09-18 23:48:44 [home_depot] INFO: [Spidermon] OK 2024-09-18 23:48:44 [scrapy.extensions.feedexport] INFO: Stored bq feed (2500 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:825 2024-09-18 23:48:45 [scrapy.extensions.feedexport] INFO: Stored bq feed (36 items) in: bq://response-elt.dev_scrapers.catalog_urls/batch:826 2024-09-18 23:48:45 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 494508, 'downloader/request_count': 257, 'downloader/request_method_count/GET': 257, 'downloader/response_bytes': 192513263, 'downloader/response_count': 257, 'downloader/response_status_count/200': 253, 'downloader/response_status_count/403': 4, 'elapsed_time_seconds': 1219.124434, 'feedexport/success_count/BigQueryFeedStorage': 826, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2024, 9, 18, 23, 48, 44, 744314, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 229112129, 'httpcompression/response_count': 160, 'httperror/response_ignored_count': 4, 'httperror/response_ignored_status_count/403': 4, 'item_dropped_count': 352, 'item_dropped_reasons_count/DropItem': 352, 'item_scraped_count': 2062536, 'log_count/ERROR': 3, 'log_count/INFO': 888, 'log_count/WARNING': 352, 'memusage/max': 1190096896, 'memusage/startup': 113750016, 'poet/injector/catalog_discovery.pages.home_depot.HomeDepotSitemapPageObject': 257, 'request_depth_max': 2, 'response_received_count': 257, 'scheduler/dequeued': 257, 'scheduler/dequeued/memory': 257, 'scheduler/enqueued': 257, 'scheduler/enqueued/memory': 257, 'spidermon/validation/fields': 8251552, 'spidermon/validation/fields/errors': 352, 'spidermon/validation/fields/errors/invalid_uri': 352, 'spidermon/validation/fields/errors/invalid_uri/url': 352, 'spidermon/validation/items': 2062888, 'spidermon/validation/items/dropped': 352, 'spidermon/validation/items/errors': 352, 'spidermon/validation/validators': 1, 'spidermon/validation/validators/item/jsonschema': True, 'spidermon_field_coverage/dict/name': 1.0, 'spidermon_field_coverage/dict/sku': 1.0, 'spidermon_field_coverage/dict/supplier': 1.0, 'spidermon_field_coverage/dict/url': 1.0, 'spidermon_item_scraped_count': 2062536, 'spidermon_item_scraped_count/dict': 2062536, 'spidermon_item_scraped_count/dict/name': 2062536, 'spidermon_item_scraped_count/dict/sku': 2062536, 'spidermon_item_scraped_count/dict/supplier': 2062536, 'spidermon_item_scraped_count/dict/url': 2062536, 'start_time': datetime.datetime(2024, 9, 18, 23, 28, 25, 619880, tzinfo=datetime.timezone.utc)} 2024-09-18 23:48:45 [scrapy.core.engine] INFO: Spider closed (finished)