2025-11-08 12:45:07 [scrapy.utils.log] (PID: 112) INFO: Scrapy 2.12.0 started (bot: catalog_extraction) 2025-11-08 12:45:07 [scrapy.utils.log] (PID: 112) INFO: Versions: lxml 5.3.1.0, libxml2 2.12.9, cssselect 1.3.0, parsel 1.10.0, w3lib 2.3.1, Twisted 24.11.0, Python 3.11.13 (main, Jun 10 2025, 23:54:42) [GCC 12.2.0], pyOpenSSL 25.0.0 (OpenSSL 3.4.1 11 Feb 2025), cryptography 44.0.2, Platform Linux-6.9.12-x86_64-with-glibc2.36 2025-11-08 12:45:07 [rocket_industrial] (PID: 112) INFO: Starting extraction spider rocket_industrial... 2025-11-08 12:45:07 [scrapy.addons] (PID: 112) INFO: Enabled addons: [] 2025-11-08 12:45:07 [py.warnings] (PID: 112) WARNING: /usr/local/lib/python3.11/site-packages/scrapy/utils/request.py:120: ScrapyDeprecationWarning: 'REQUEST_FINGERPRINTER_IMPLEMENTATION' is a deprecated setting. It will be removed in a future version of Scrapy. return cls(crawler) 2025-11-08 12:45:07 [scrapy.extensions.telnet] (PID: 112) INFO: Telnet Password: 2c3b33379c696016 2025-11-08 12:45:07 [py.warnings] (PID: 112) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-08 12:45:07 [scrapy.middleware] (PID: 112) INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.closespider.CloseSpider', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats', 'spidermon.contrib.scrapy.extensions.Spidermon'] 2025-11-08 12:45:07 [scrapy.crawler] (PID: 112) INFO: Overridden settings: {'BOT_NAME': 'catalog_extraction', 'CONCURRENT_ITEMS': 250, 'FEED_EXPORT_ENCODING': 'utf-8', 'LOG_FILE': '/var/lib/scrapyd/logs/catalog_extraction/rocket_industrial/bd82267cbca011f08d4f4200a9fe0102.log', 'LOG_FORMAT': '%(asctime)s [%(name)s] (PID: %(process)d) %(levelname)s: ' '%(message)s', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'catalog_extraction.spiders', 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_poet.ScrapyPoetRequestFingerprinter', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'SPIDER_MODULES': ['catalog_extraction.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor', 'USER_AGENT': None} 2025-11-08 12:45:08 [scrapy_poet.injection] (PID: 112) INFO: Loading providers: [, , , , , , ] 2025-11-08 12:45:08 [scrapy.middleware] (PID: 112) INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scraping_utils.middlewares.downloaders.ProxyManagerDownloaderMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scraping_utils.middlewares.downloaders.HeadersSpooferDownloaderMiddleware', 'scrapy_poet.InjectionMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy_poet.DownloaderStatsMiddleware'] 2025-11-08 12:45:08 [NotFoundHandlerSpiderMiddleware] (PID: 112) INFO: NotFoundHandlerSpiderMiddleware running on PRODUCTION environment. 2025-11-08 12:45:08 [scrapy.middleware] (PID: 112) INFO: Enabled spider middlewares: ['catalog_extraction.middlewares.NotFoundHandlerSpiderMiddleware', 'catalog_extraction.middlewares.FixtureSavingMiddleware', 'scrapy_poet.RetryMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-11-08 12:45:08 [scrapy.middleware] (PID: 112) INFO: Enabled item pipelines: ['catalog_extraction.pipelines.DuplicatedSKUsFilterPipeline', 'catalog_extraction.pipelines.DiscontinuedProductsAdjustmentPipeline', 'catalog_extraction.pipelines.PriceRoundingPipeline', 'scraping_utils.pipelines.AttachSupplierPipeline', 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline'] 2025-11-08 12:45:08 [scrapy.core.engine] (PID: 112) INFO: Spider opened 2025-11-08 12:45:08 [scrapy.extensions.closespider] (PID: 112) INFO: Spider will stop when no items are produced after 1800 seconds. 2025-11-08 12:45:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2025-11-08 12:45:08 [scrapy.extensions.telnet] (PID: 112) INFO: Telnet console listening on 127.0.0.1:6025 2025-11-08 12:45:10 [ProxyManagerDownloaderMiddleware] (PID: 112) INFO: Using brd-customer-hl_13cda1e4-zone-sharedpool_datacenter_proxy as the default proxy for ProxyManagerDownloaderMiddleware. 2025-11-08 12:45:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-wat-m-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-8-inch-x-024-x-12-900-machine-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacmaster-vp210-tabletop-chamber-vacuum-sealer-maintenance-free.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ipg-stretchflex-sf1-90-gauge-stretch-wrap.html returned 404 status code. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-8-inch-x-024-x-12-900-8-8-core-machine-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-x-024-x-7900-8-8-core-machine-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/industrial-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-x-031-x-7-200-hand-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/combi-replacement-blades.html returned 404 status code. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/legend-series-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-x-024-x-9-900-machine-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-7000a-pro-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-14-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-2-ez-high-speed-drop-packer-integrated-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-6-x-7-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-x-027-x-7200-hand-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/samuel-p702-arch-banding-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-9-000-black-polypropylene-hand-strapping-300-lb-16-x-6-core.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/seal-a-tron-food-grade-stainless-steel-heat-tunnel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-x-025-x-7-200-black-pp-hand-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-18-160-edge-protector.html returned 404 status code. 2025-11-08 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-023-9900-clear-polypropylene-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-8-x-023-12900-white-polypropylene-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b600-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/better-pack-755esa-water-activated-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/better-pack-industrial-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/better-pack-tape-machine-with-heater.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/curby-mini-taper-water-activated-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/better-pack-555esa-water-activated-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/phoenix-e1-water-activated-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/20-inch-extended-core-stretch-wrap.html returned 404 status code. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/better-pack-shipping-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/phoenix-e4-water-activated-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/phoenix-e2-water-activated-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/anti-static-esd/anti-static-bubble-packaging/1-2-x-24-static-control-bubble-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/better-pack-packer-3s-water-activated-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-wat-e-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-3-160-strapping-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-x-75-gauge-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/resealable-metallic-static-shielding-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/50ft-soft-wire-19ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/large-resealable-white-block-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/food-packaging/vacuum-packing/vacuum-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-6-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-6-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/food-packaging/vacuum-packing/vacuum-pouches.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-30-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/markable-white-block-closable-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-11-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-32-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/flair-18-x-28-5-mil-standard-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-34-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-18-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-13-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-5-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/flair-14-x-24-5-mil-standard-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/catalog/product/view/id/24950/s/6-x-12-flairpak-400/category/197/ already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-11-standard-vacuum-pouch-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/chamber-vacuum-bag-12x22-5mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-28-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-54-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-36-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-14-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/38-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-25-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-18-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/food-packaging/vacuum-packing/vacuum-pouches.html?p=3 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-22-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-30-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-10-x-24-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-4-x-20case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-15-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-15-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-22-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/food-packaging/vacuum-packing/vacuum-pouches.html?p=4 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-26-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-22-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-22-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-26-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-15-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-20-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/food-packaging/vacuum-packing/vacuum-pouches.html?p=5 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-18-x-36-gusseted-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-26-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-28-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-25-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-20-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-15-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heavy-carton-splicing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-13-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-14-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-18-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/red-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-25-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-24-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:45:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-14-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-18-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-22-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-inch-friction-reduction-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-22-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/food-packaging/vacuum-packing/vacuum-pouches.html?p=5 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-14-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nopi-4287-strapping-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacuum-pouch-16-x-20-4mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-5-in-x-60-yd-filament-tape-rg316.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-15-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-in-x-60-yd-filament-tape-rg300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/start-automatic-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacmaster-vp220-commercial-chamber-vacuum-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 238 pages (at 238 pages/min), scraped 158 items (at 158 items/min) 2025-11-08 12:46:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-60-yd-filament-tape-rg300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ocme-auriga-ct-counterbalanced-automated-guided-vehicle.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-15-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/silver-duct-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-22-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-22-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-32-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-110-yd-clear-box-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ocme-auriga-ct-counterbalanced-automated-guided-vehicle.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-200l-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-lp-3200-low-profile-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-tape-hub-assembly.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/berran-t-100-belt-set.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-cylinder-n401-358.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-zippered-poly-bags-4-x-6-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-48-single-wall-corrugated-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sharp-max-pro-24-roll-bagging-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-caster-wheel-ldc302.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/paper-film-manufacturing-converting.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/automation-social-distancing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-potentiometer-fg-poten.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-10-x-14-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-mbd-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-drive-wheel-sleeve-fjg-1a-208.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:23 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/coronavirus-and-packaging returned 404 status code. 2025-11-08 12:46:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-48-x-42-x-60-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/electrical-tape-vs-duct-tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/5-misconceptions-about-the-humble-corrugated-box already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/paper-film-manufacturing-converting.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/paper-film-manufacturing-converting.html landed on page that is not a product page. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/what-is-box-makers-certificate already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/automation-social-distancing.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/automation-social-distancing.html landed on page that is not a product page. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-inch-open-snap-on-poly-strapping-seals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-polychem-textured-strapping-seal.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-polychem-textured-strapping-seal.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-1-steel-snap-on-seals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/small-box-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/case-sealers-erectors/case-sealers/uniform-case-sealers.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-100l-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t210l-large-box-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t150-case-sealer-with-top-flap-folder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/electrical-tape-vs-duct-tape) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/electrical-tape-vs-duct-tape landed on page that is not a product page. 2025-11-08 12:46:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/5-misconceptions-about-the-humble-corrugated-box) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/5-misconceptions-about-the-humble-corrugated-box landed on page that is not a product page. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-210-carton-taping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/case-sealer-buyers-guide already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100ff-uniform-case-sealer-flap-folder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t200-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/what-is-box-makers-certificate) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/what-is-box-makers-certificate landed on page that is not a product page. 2025-11-08 12:46:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/start-at-the-end-to-understand-the-beginning returned 404 status code. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-at4e-automatic-uniform-four-edges-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-3-120-strapping-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wexxar-bel-290-high-speed-automatic-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/case-sealer-starter-package.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/abal-centurion-case-sealer-by-loveshaw.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wexxar-bel-250-uniform-automatic-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/make-your-packaging-operation-a-great-place-to-work returned 404 status code. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/case-sealers-erectors/case-sealers/uniform-case-sealers.html?p=3 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/compact-automatic-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/automatic-side-drive-box-closer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/little-david-trade-ld-xss-rte-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/case-sealer-buyers-guide) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/case-sealer-buyers-guide landed on page that is not a product page. 2025-11-08 12:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/box-erecting-pack-tape-station.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/maximizing-efficiency-and-protection-with-stretch-film-roping already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/quad-drive-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:26 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/100-anniversary-of-united-paper returned 404 status code. 2025-11-08 12:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/t-rail-bottom-folding-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/top-bottom-drive-carton-taper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-2024-side-belt-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-200a-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/legend-series-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-bottom-belt-case-taper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-side-drive-carton-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ld-7-pressure-sensitive-case-taper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tite-dri-classic-4-inch-x-7-inch-black-white-meat-fish-poultry-absorbent-pad-75-grams.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-usa-20-sb-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/maximizing-efficiency-and-protection-with-stretch-film-roping) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/maximizing-efficiency-and-protection-with-stretch-film-roping landed on page that is not a product page. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eastey-reg-bb-2-bottom-belt-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t500l-uniform-case-sealer-flap-folder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-7000a3-pro-adjustable-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-8000a-adjustable-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-8000af3-adjustable-flap-folding-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-800ab3-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-8000af-adjustable-flap-folding-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ld-16a-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-trade-ms1e-one-edge-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/automatic-four-belt-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/compact-uniform-automatic-taper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100ss-stainless-steel-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-500-uniform-case-sealer-and-box-flap-folder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bottom-sealing-box-taper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100b-bottom-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/simple-pack-seal-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wexxar-bel-252-uniform-automatic-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-mtd22-uniform-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wexxar-bel-150-semi-automatic-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-100-carton-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-inch-serrated-seals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-serrated-seals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-open-snap-on-poly-strapping-seals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-open-snap-on-poly-strapping-seals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-side-drive-carton-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-210-carton-taping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-inch-open-snap-on-poly-strapping-seals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-100-carton-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-48-160-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/light-duty-ratchet-and-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heavy-duty-strap-tensioning-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-24-160-edge-protector.html returned 404 status code. 2025-11-08 12:46:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ipg-20-5830-45-gauge-exlfilmplus-shrink-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-12-120-edge-protector.html returned 404 status code. 2025-11-08 12:46:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/limit-switch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/carriage-down-relay.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-control-board.html returned 404 status code. 2025-11-08 12:46:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-air-pillow-8-by-4-inch-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/35lb-vci-kraft-paper-roll-48-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/premier-protective-packaging-white-polyethylene-foam-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-k19-vbelt.html returned 404 status code. 2025-11-08 12:46:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/superflex-63-gauge-long-roll.html returned 404 status code. 2025-11-08 12:46:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-1-2-inch-phosphate-coated-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/17-5-inch-spartan-59-gauge-hand-stretch-film-1500-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-air-pillow-8-by-4-inch-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-4375-intertape-60-gauge-exlfilmplus-gps-pre-perfed-shrink-wrap-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:45 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/35lb-vci-poly-coated-kraft-paper-roll-48-x-200yd.html returned 404 status code. 2025-11-08 12:46:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/detroit-forming-ops-plastic-clear-locking-hinged-pie-container.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/resealable-pink-antistatic-bag.html returned 404 status code. 2025-11-08 12:46:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dw-fine-pack-half-sheet-cake-aluminum-pan.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/synergy-high-profile-turntable.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-clear-resealable-bag-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/air-powered-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-pre-opened-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-8-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-7-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-6-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:46:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-30-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-24-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-14-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-5-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-30-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-32-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-40-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-30-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-60-yds-duct-tape-36.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/pink-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-3-x-15-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-42-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 495 pages (at 257 pages/min), scraped 326 items (at 168 items/min) 2025-11-08 12:47:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/two-inch-metal-filament-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/replacement-kit-8-inch-poly-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-ecoplat-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/toptier-conventional-low-infeed-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-6425-l-clip-tape-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-main-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/orgapack-ort-260-handheld-banding-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-708-rotary-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-motor-cv200-20zg1-g2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-6100ss-automatic-definite-length-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-secunorm-smartcut-mdp-food-grade-safety-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-secumax-150-mdp-food-grade-safety-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/saint-gobain-200a-silicone-sponge-tape-36-x-10.html returned 404 status code. 2025-11-08 12:47:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-51-x-46-x-88-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/start-international-upper-lower-blade-set-for-zcm1000.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-5-inch-orange-invoice-enclosed-envelope.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cac61-tape-head-loveshaw.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-secunorm-smartcut-mdp-food-grade-safety-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-18-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/carry-handles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/packaging-health-assessment.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/gilberts.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-12-x-18-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-60-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/amazon-packaging-waste returned 404 status code. 2025-11-08 12:47:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/packaging-health-assessment.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/packaging-health-assessment.html landed on page that is not a product page. 2025-11-08 12:47:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-finger-plate-shaft.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/carry-handles.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/carry-handles.html landed on page that is not a product page. 2025-11-08 12:47:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/gilberts.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/food-wrap-paper-comparison-guide already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/table-top-poly-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/materials already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/double-tamp-label-printer-and-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/packaging-social-media-accounts-you-need-to-follow already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/10-facts-about-packaging-that-will-impress-your-friends returned 404 status code. 2025-11-08 12:47:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/gilberts.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-9-x-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-matthew-bruss returned 404 status code. 2025-11-08 12:47:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-inch-bundling-hand-stretch-film-90-ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/category/materials) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/category/materials landed on page that is not a product page. 2025-11-08 12:47:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/packaging-social-media-accounts-you-need-to-follow) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/packaging-social-media-accounts-you-need-to-follow landed on page that is not a product page. 2025-11-08 12:47:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/contact-us/gilberts.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/contact-us/gilberts.html landed on page that is not a product page. 2025-11-08 12:47:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/superflex-50-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/food-wrap-paper-comparison-guide) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/food-wrap-paper-comparison-guide landed on page that is not a product page. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-sp11-9-x-8-core-specialty-tabletop-banding-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-pad-paper-cushioning-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-sp11-tabletop-bander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-inch-nexus-machine-film-55-gauge-stretch-wrap-8000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-5000-intertape-95-gauge-genesys-superflex-machine-stretch-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/semi-automatic-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-pc101-semi-automatic-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mosca-mom-8-tabletop-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-40-120-edge-protector.html returned 404 status code. 2025-11-08 12:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/expandos-md-high-performance-recyclable-packing-material.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-16-x-12-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/samuel-p650-tabletop-banding-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/deli-container-lids.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/25ft-half-hard-wire-18ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-power-switch.html returned 404 status code. 2025-11-08 12:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/secumax-150.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-microswitch.html returned 404 status code. 2025-11-08 12:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-slider-top-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dyne-a-pak-8-5-inch-x-4-5-inch-black-foam-trays.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/evolution-2-high-resolution-industrial-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-resealable-bubble-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-5-clear-resealable-bag-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/secumax-150.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/airmove-void-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-6-x-20-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/airmover-replacement-bubble-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/rocket-industrial-air-pillow-8-by-4-inch-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-4-lb-red-plaid-paper-food-tray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/rocket-industrial-inflatable-bubble-16-by-10-inch-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/polyair-airspace-air-pillow-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/polyair-airspace-bubble-on-demand-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/airmove-bubble-cushioning-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/airmove-wrap-cushioning-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/resealable-transparent-antistatic-bag.html returned 404 status code. 2025-11-08 12:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-slider-top-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/airmove-cushion-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/foot-pedal-for-airmove2-air-pillow-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-10-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-10-x-24-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-clear-poly-tubing-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/spare-parts-kit-airmove2-air-pillow-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/air-pillows/air-pillows-packs/airmove-void-air-pillows.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-4-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-9-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-3-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-26-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/10-x-15-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 12:47:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-60-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/4-x-7-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 12:47:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-squeezer-wheel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-1000yd-3m-311-plus-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/packing-table-for-ld7-ld19.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-25-x-12-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-23-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-black-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-8100-heavy-duty-clear-machine-length-packaging-tape-3-x-1000-yards-2-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/clear-poly-bin-liner-26-16-32-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-handheld-strapping-tool-1-2-inch-5-8-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-60-yd-flatback-paper-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/texwrap-st-3322-automatic-in-line-l-bar-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-venom-water-activated-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-black-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/5-5-inch-orange-face-invoice-envelope.html returned 404 status code. 2025-11-08 12:47:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-endless-56-5-inch-belt-set.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacmaster-vp400-commercial-double-chamber-vacuum-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/major-minor-box-top-folder-for-3-in-ld3-ld7-ld19.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-evolution-lt-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-alr-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-kl-150-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-l-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-x-023-standard-grade-steel-strapping-2070-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dual-temperature-heat-gun.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:45 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <403 https://www.rocketindustrial.com/robopac-compacta-4-orbital-stretch-wrapper.html>: HTTP status code is not handled or not allowed 2025-11-08 12:47:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-25-x-12-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-alr-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-lt-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:46 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/taconic-6085-06-ptfe-fiberglass-cloth-tape-39-x-36.html returned 404 status code. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-csd-automatic-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-order-picking-platform-trailer-42x72.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3m-5490-ptfe-extruded-film-tape-12-x-36.html returned 404 status code. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-order-picking-platform-42x48.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-axle-sleeve-fj-02-10.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/synergy-low-profile-turntable-demo.html returned 404 status code. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-cseg-egg-automatic-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-tartan-369-3-inch-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-portable-dandy-lift-table-220-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-36-powered-turntable-2000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/become-a-vendor.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-3-x-5-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/create-your-own-internship.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/product-id-marking-fiber-case-study.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-tartan-369-3-inch-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/tulsa.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/difference-between-burst-edge-crush-test already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/equipment-rebate-credit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-ps800-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:49 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/create-your-own-internship.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/create-your-own-internship.html landed on page that is not a product page. 2025-11-08 12:47:50 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/become-a-vendor.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/become-a-vendor.html landed on page that is not a product page. 2025-11-08 12:47:50 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/product-id-marking-fiber-case-study.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/product-id-marking-fiber-case-study.html landed on page that is not a product page. 2025-11-08 12:47:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-august already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/tulsa.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/stretch-wrapper-buying-guide already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microjet-hrp-controller-package.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/difference-between-burst-edge-crush-test already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/request-stretch-wrapping-video-demo.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/spring-cleaning-tips-for-commercial-facilities already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-pc1000-automatic-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-pc1000-automatic-strapping-machine-1-2-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/samuel-p702rs-arch-banding-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-710-arch-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/equipment-rebate-credit.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/equipment-rebate-credit.html landed on page that is not a product page. 2025-11-08 12:47:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-august) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-august landed on page that is not a product page. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/strapping-kits.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-woven-strap-starter-kit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/value-postal-approved-strapping-kit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/felins-us-2000-ad-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/contact-us/tulsa.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/contact-us/tulsa.html landed on page that is not a product page. 2025-11-08 12:47:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/stretch-wrapper-buying-guide) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/stretch-wrapper-buying-guide landed on page that is not a product page. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-710l-arch-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mosca-romp-6-strapping-machine-with-sonixs-sealing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mosca-romp-6b-strapping-machine-with-sonixs-sealing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/difference-between-burst-edge-crush-test) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/difference-between-burst-edge-crush-test landed on page that is not a product page. 2025-11-08 12:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mosca-romp-6r-strapping-machine-with-sonixs-sealing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/request-stretch-wrapping-video-demo.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/request-stretch-wrapping-video-demo.html landed on page that is not a product page. 2025-11-08 12:47:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/spring-cleaning-tips-for-commercial-facilities) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/spring-cleaning-tips-for-commercial-facilities landed on page that is not a product page. 2025-11-08 12:47:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/anser-u2-handheld-mobile-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mosca-roms-6-strapping-machine-with-sonixs-sealing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mosca-rom-fusion-semi-automatic-arch-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-s900-arch-strapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/strapping-solutions-for-the-lumber-industry already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:53 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/five-takeaways-spc-conference returned 404 status code. 2025-11-08 12:47:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/evolution-4711bk-fast-drying-black-ink.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microjet-hrp-non-porous-ink.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fast-drying-u2-ink-cartridge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/premium-hrp-ink-cartridges.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/matthews-cth-1000-black-ink-cartridges.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/matthews-cth-1300-black-ink-cartridges.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/standard-u2-ink-cartridge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/evolution-4500bk-black-ink-cartridge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/strapping-solutions-for-the-lumber-industry) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/strapping-solutions-for-the-lumber-industry landed on page that is not a product page. 2025-11-08 12:47:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sp4-1-black-ink-cartridge-for-the-anser-u2-smartone-1-inch-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-replacement-ink-608.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/strapping-break-strength-calculator already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-48-120-edge-protector.html returned 404 status code. 2025-11-08 12:47:58 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-18-120-edge-protectors.html returned 404 status code. 2025-11-08 12:47:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sp4-1-black-ink-cartridge-for-the-anser-u2-smartone-1-inch-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:47:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/strapping-break-strength-calculator) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/strapping-break-strength-calculator landed on page that is not a product page. 2025-11-08 12:48:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-heavy-duty-metal-strapping-buckle.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stretch-dancer-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fill-paper-void-fill-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/megasafe-standard-use-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-3-160-strapping-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/versapak-12-oz-clear-hinged-pet-plastic-clamshell-flat-lid.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-39-37-in-pack-exit-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000eb-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000be-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:03 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/30-inch-hand-held-stretch-wrap.html returned 404 status code. 2025-11-08 12:48:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-24-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-robotic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microjet-i-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/steel-box-cutter-utility-knife.html returned 404 status code. 2025-11-08 12:48:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/best-pack-random-side-drive-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ep-810-t-handle-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/anti-static-esd/static-shielding-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-trusted-lock-freezer-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-shorty-brown-paper-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 854 pages (at 359 pages/min), scraped 503 items (at 177 items/min) 2025-11-08 12:48:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/electronic-safe-resealable-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/metallic-static-control-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/large-resealable-metallic-shielding-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-5-static-shielding-layflat-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-static-shielding-layflat-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-static-shielding-layflat-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-static-shielding-layflat-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/variable-box-sealing-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-layflat-conductive-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-8-x-24-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-48-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-26-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-2-x-12case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-26-x-60-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-371-2-x-1000-yard-carton-sealing-tape-case-of-6-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-x-60-yds-masking-tape-intertape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-knife-arm-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-28-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ld19-belt-set.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-coarse-tooth-tape-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-case-sealer-3-flap-folder-um869.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b1000-handheld-battery-powered-strapping-tool-batteries-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/infeed-assisting-and-packaging-table-ldu.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-automatic-twist-tie-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/8-x-10-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 12:48:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-6-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-automatic-twist-tie-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-wide-impulse-plastic-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-dancer-bar-fg-142.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-321-medium-duty-blue-acrylic-packaging-tape-2-x-110-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:28 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/18-5-x-11-875-curby-mailer.html returned 404 status code. 2025-11-08 12:48:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-8-x-10-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vc2-2-vci-vapor-capsules.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg18-packaging-tape-2-x-110.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/matthews-t-series-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-rs20-random-bottom-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-spkt-ldx-61.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/plastic-banding-tool-orgapack-ort-55.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-100-clear-poly-sheeting-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ecommerce-packaging-solutions.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-limit-switch-ldf507.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/account-business-lander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-coarse-tooth-loveshaw-tape-blade-m2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b1000-handheld-battery-powered-strapping-tool-batteries-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-vacuum-cup-oval-vc-1012.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/improve-packaging-productivity-with-conveyors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:30 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/creative-packaging-mattress-industry returned 404 status code. 2025-11-08 12:48:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/packaging-optimization.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:30 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ecommerce-packaging-solutions.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ecommerce-packaging-solutions.html landed on page that is not a product page. 2025-11-08 12:48:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog landed on page that is not a product page. 2025-11-08 12:48:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/account-business-lander.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/account-business-lander.html landed on page that is not a product page. 2025-11-08 12:48:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/3m-products-close-by returned 404 status code. 2025-11-08 12:48:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/cast-vs-blown-stretch-wrap-whats-the-difference already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-jen-rybicki returned 404 status code. 2025-11-08 12:48:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/improve-packaging-productivity-with-conveyors) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/improve-packaging-productivity-with-conveyors landed on page that is not a product page. 2025-11-08 12:48:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/shrink-wrapping/shrink-wrap-accessories/seal-pads.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/2022-year-in-review returned 404 status code. 2025-11-08 12:48:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/packaging-optimization.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/packaging-optimization.html landed on page that is not a product page. 2025-11-08 12:48:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-40-160-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-tow200-ss-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/art-of-packaging already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:33 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/cast-vs-blown-stretch-wrap-whats-the-difference) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/cast-vs-blown-stretch-wrap-whats-the-difference landed on page that is not a product page. 2025-11-08 12:48:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-72-120-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:33 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/preparing-workplaces-beyond-covid-19 returned 404 status code. 2025-11-08 12:48:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dancer-proximity-sensor.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-24-225-edge-protector.html returned 404 status code. 2025-11-08 12:48:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-genesys-80-gauge-eq-stretch-wrap-5000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10yd-sponge-pad.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/17-inch-torque-38-gauge-hand-stretch-film-2000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:37 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/art-of-packaging) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/art-of-packaging landed on page that is not a product page. 2025-11-08 12:48:37 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/18-inch-green-stretch-wrap.html returned 404 status code. 2025-11-08 12:48:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-oz-single-serving-deli-container.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fox-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fox-fps-400h-high-profile-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-poly-tubing-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-genesys-80-gauge-eq-stretch-wrap-5000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-36-120-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-rsa-3036-sb-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/anti-static-esd/static-shielding-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-36-gusseted-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:42 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 12:48:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:42 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/20-inch-genesys-80-gauge-eq-stretch-wrap-5000-foot-rolls.html>: HTTP status code is not handled or not allowed 2025-11-08 12:48:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-20-x-48-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-8-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-3-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-4-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/50ft-soft-wire-18ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-4-x-3-4-gray-silicone-sponge-seal-10-yard-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-5-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/orange-silicone-sponge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/silicone-sponge-pad.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/silicone-sponge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10yd-silicone-seal-pad.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-5-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-10-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-8-x-30-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-8-x-24case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-3-x-15case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-9-x-24case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-40-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-8-x-24-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-40-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-40-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-60-yd-blue-painters-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-hot-melt-glue-sticks-cold-resistant-5-8-inch-10-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-1000-yard-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/34-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-in-x-60-yd-intertape-534-flatback-paper-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-60-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cac1500-abal-tape-cartridge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/box-top-pressure-applicator-ld16a.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-peach-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-earth-epphl-66-carryout-food-containers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2-inch-tape-head-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/10-x-12-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 12:48:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-1-2-x-10-poly-bubble-mailers-b831.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-upm1161-drive-belt.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-masterplat-stretch-wrapper-86-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-green-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-roller-bearing-5972k168.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-strapping-tool-replacement-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-solenoid-valve-n402-133-24vdc.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-vacuum-cup-oval-vc-1013.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2-inch-tape-head-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:48:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-bronze-flange-bushing-50186-039.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-door-latch-fg-lock2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/6-in-packing-list-enclosed-red-face.html returned 404 status code. 2025-11-08 12:49:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/bestpack-bg18-machine-length-tape-2-x-1000.html returned 404 status code. 2025-11-08 12:49:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/columbia-machine-fl1000-floor-level-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-bronze-flange-bushing-50186-039.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-zls-low-profile-lift-table-4000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-tow200-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/columbia-machine-hl2200-high-level-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-18-flexible-skate-wheel-conveyor.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ri-200-strapping-gear-cover.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/protective-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/sustainable-packaging-solution.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-mt400-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/selecting-a-case-sealer.html returned 404 status code. 2025-11-08 12:49:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-m8-t-nut.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-plain-white-carry-handles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/sustainable-packaging-solution.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/sustainable-packaging-solution.html landed on page that is not a product page. 2025-11-08 12:49:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-wat-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/protective-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/protective-packaging.html landed on page that is not a product page. 2025-11-08 12:49:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-2ez-xl-case-erector-with-bottom-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-february already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/xtravac-cm800l.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/xtravac-cm360lr.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:06 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-josh-struck returned 404 status code. 2025-11-08 12:49:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/xtravac-cm430.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:06 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/brown-is-green-story-of-corrugated-recycling-success returned 404 status code. 2025-11-08 12:49:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacmaster-vp320-commercial-chamber-vacuum-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/xtravac-cm300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacmaster-vp215-commercial-chamber-vacuum-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacmaster-vp600-commercial-double-chamber-vacuum-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-60-120-edge-protector.html returned 404 status code. 2025-11-08 12:49:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 1177 pages (at 323 pages/min), scraped 653 items (at 150 items/min) 2025-11-08 12:49:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vacmaster-vp800-commercial-double-chamber-vacuum-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:09 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/5-uses-vr-manufacturing returned 404 status code. 2025-11-08 12:49:09 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-february) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-february landed on page that is not a product page. 2025-11-08 12:49:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ri-200-strapping-tool-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-clear-resealable-bag-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-nexus-machine-film-55-gauge-stretch-wrap-8000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-nexus-machine-film-60-gauge-stretch-wrap-7500-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-30-120-edge-protector.html returned 404 status code. 2025-11-08 12:49:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-lift-tables-can-improve-warehouse-efficiency already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fromm-replacement-charger-p328-p329.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/free-standing-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eco-friendly-packing-peanuts.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/20-x-800-goodwrappers-120-gauge-economy-hand-stretch-film.html returned 404 status code. 2025-11-08 12:49:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/30-inch-extended-core-stretch-wrap.html returned 404 status code. 2025-11-08 12:49:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/18-inch-red-stretch-wrap.html returned 404 status code. 2025-11-08 12:49:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/edgetec-200-l-clip-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-lift-tables-can-improve-warehouse-efficiency) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-lift-tables-can-improve-warehouse-efficiency landed on page that is not a product page. 2025-11-08 12:49:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-028-x-6500-green-polyester-smooth-tool-grade-strapping-coil-785-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wipe-on-labeler.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/brushless-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-trusted-guard-freezer-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/extra-small-resealable-white-block-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-oz-side-dish-deli-container.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-flat-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-55-gauge.html returned 404 status code. 2025-11-08 12:49:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/pink-resealable-antistatic-bag.html returned 404 status code. 2025-11-08 12:49:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-8-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-18-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-5-x-2-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-6-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-6-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-14-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-15-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-5-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-14-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-5-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-8-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-3-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-5-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-9-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-7-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-36-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-60-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-4-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-42-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-16-x-60case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-10-x-32case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-32-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-24-x-48-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-30-peach-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-12-x-36-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-duct-tape-43.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-16-x-42-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1500-yard-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/7-x-10-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 12:49:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-interpack-standard-tape-head.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-2-inch-fine-tooth-blade-for-cac60-tape-head.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-14-x-36-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-10-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-5-1-2-self-seal-bubble-pouches-bob45.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-28-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-case-sealer-replacement-drive-belt.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-high-temp-hot-melt-glue-gun-1-2-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/standard-tape-gun.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-stand-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/madison-st-3-automatic-orbital-bander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/white-gaffers-tape-low-gloss-finish.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/maintenance-kit-12-inch-pouch-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-01-230-black-twist-tie-ribbon.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/standard-interpack-tape-head.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-screw-retainer-assembly-closed-fj-sr-c-assy.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-zippered-poly-bags-6-x-8-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-order-picking-platform-trailer-48x96.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/high-speed-kit-for-ldu-ldr.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-fine-tooth-tape-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/our-capabilities.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-01-230-black-twist-tie-ribbon.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/grocery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/our-mission.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/horizontal-kraft-paper-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/preventative-maintenance-program.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ri-200-strapping-tension-cover.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/grocery.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/grocery.html landed on page that is not a product page. 2025-11-08 12:49:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/packlytics-stretch-wrap-monitoring-system.html returned 404 status code. 2025-11-08 12:49:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/what-is-vci-and-how-does-it-work already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:55 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-75-mailing-tube-end-caps.html returned 404 status code. 2025-11-08 12:49:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/our-mission.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/our-mission.html landed on page that is not a product page. 2025-11-08 12:49:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100rb-bottom-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/preventative-maintenance-program.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/preventative-maintenance-program.html landed on page that is not a product page. 2025-11-08 12:49:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/our-capabilities.html returned 404 status code. 2025-11-08 12:49:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/reinvigorating-american-manufacturing returned 404 status code. 2025-11-08 12:49:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-to-increase-the-longevity-of-your-banding-tool already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/what-is-vci-and-how-does-it-work) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/what-is-vci-and-how-does-it-work landed on page that is not a product page. 2025-11-08 12:49:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/you-shouldnt-be-totally-afraid-of-automation returned 404 status code. 2025-11-08 12:49:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-to-maintain-case-sealer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-rubber-drive-wheel-fj-1a-108.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:58 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-katie-zoborowski returned 404 status code. 2025-11-08 12:49:58 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/battery-powered-steel-strapping-sealer-tensionser-kits.html returned 404 status code. 2025-11-08 12:49:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-industrial-is-great-place-to-work-certified-for-2022 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-amtopp-75-gauge-high-performance-stretch-wrap-6000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/lastest-in-pet-food-packaging-trends already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:58 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/strapping-pallet-feeder-tool.html returned 404 status code. 2025-11-08 12:49:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-to-increase-the-longevity-of-your-banding-tool already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-to-maintain-case-sealer) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-to-maintain-case-sealer landed on page that is not a product page. 2025-11-08 12:49:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-industrial-is-great-place-to-work-certified-for-2022) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-industrial-is-great-place-to-work-certified-for-2022 landed on page that is not a product page. 2025-11-08 12:49:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/lastest-in-pet-food-packaging-trends) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/lastest-in-pet-food-packaging-trends landed on page that is not a product page. 2025-11-08 12:49:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-to-increase-the-longevity-of-your-banding-tool) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-to-increase-the-longevity-of-your-banding-tool landed on page that is not a product page. 2025-11-08 12:49:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:49:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-115-gauge.html returned 404 status code. 2025-11-08 12:49:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-6-x-24-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/battery-for-orgapack-400-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-inch-phosphate-coated-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-48-vboard-edge-protectors.html returned 404 status code. 2025-11-08 12:50:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-15-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-8-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-6-x-24-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-48-160-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/metal-scraper-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 1464 pages (at 287 pages/min), scraped 775 items (at 122 items/min) 2025-11-08 12:50:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-6-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-x-48-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-2-x-8case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-6-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/centerfolded-poly-sheeting-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-in-x-60-yd-masking-tape-intertape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-x-24case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-55-yd-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-nylon-bushing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-5-x-13-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-60-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/plastic-film-shrink-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wheels-loveshaw-sp304.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/34-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/premium-poly-strapping-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/astro-ls10-hot-melt-tank.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-lcd-screen-controller.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-inch-x-18-yd-ptfe-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-main-motor-ys7124-b14.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-in-x-110-yd-cold-temp-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-roll-holder-assembly-fg-03a-20assy.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-tensioning-shaft-fjg-1a-141.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-yellow-vinyl-tape-1-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-idler-wheel-fg-01a-21.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fromm-p329-friction-weld-banding-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/good-natured-70-gauge-hand-stretch-wrap.html returned 404 status code. 2025-11-08 12:50:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-54-x-44-x-96-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-white-vinyl-tape-2-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polypropylene-polyester-front-serratable-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-1000-white-unwaxed-butcher-paper-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/air-bubble-rolls-3-16-12-300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/case-sealer-automation.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/about-the-company already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-e-z-reach-portable-container-tilter-4000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wexxar-wf20-fully-automatic-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/no-end-labor-shortages returned 404 status code. 2025-11-08 12:50:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/parts-kit-12-inch-tubing-closer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/robots-tech-take-over-pyeongchang returned 404 status code. 2025-11-08 12:50:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-40-x-28-x-60-3-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/interview-packaging-engineer returned 404 status code. 2025-11-08 12:50:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/section-179-tax-deduction-for-packaging-equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-e-z-reach-portable-container-tilter-4000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/about-the-company) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/about-the-company landed on page that is not a product page. 2025-11-08 12:50:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/case-sealer-automation.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/case-sealer-automation.html landed on page that is not a product page. 2025-11-08 12:50:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-compacta-9-orbital-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-compacta-12-orbital-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/chamber-vacuum-sealer-buyers-guide already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-e-z-reach-portable-container-tilter-4000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:21 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/section-179-tax-deduction-for-packaging-equipment) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/section-179-tax-deduction-for-packaging-equipment landed on page that is not a product page. 2025-11-08 12:50:21 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/helping-the-heroes-of-covid19 returned 404 status code. 2025-11-08 12:50:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/top-companies-cutting-down-on-packaging-waste-and-costs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-023-standard-grade-steel-strapping-1380-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/best-practices-for-storing-adhesive-tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:22 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/chamber-vacuum-sealer-buyers-guide) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/chamber-vacuum-sealer-buyers-guide landed on page that is not a product page. 2025-11-08 12:50:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-november already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:23 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-12-120-edge-protectors.html returned 404 status code. 2025-11-08 12:50:23 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-60-160-edge-protector.html returned 404 status code. 2025-11-08 12:50:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/top-companies-cutting-down-on-packaging-waste-and-costs) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/top-companies-cutting-down-on-packaging-waste-and-costs landed on page that is not a product page. 2025-11-08 12:50:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-72-160-edge-protector.html returned 404 status code. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-x-023-standard-grade-steel-strapping-1725-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-hp-resistor.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/best-practices-for-storing-adhesive-tape) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/best-practices-for-storing-adhesive-tape landed on page that is not a product page. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-cushioning/expandos-md-high-performance-recyclable-packing-material.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-cushioning/versa-pak-dispenser-box.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-machine-film-80-gauge-stretch-wrap-6000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stretchflex-63-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-16-x-24-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-november) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-november landed on page that is not a product page. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-inch-trusted-ultra-guard-locker-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/small-resealable-bag-with-a-white-block.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-x-029-high-tensile-grade-steel-strapping-3350-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/samuel-end-gripper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-8-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-10-x-30-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-plastic-strapping-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-slider-top-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/samuel-end-gripper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stretch-chain.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/transparent-white-c-a-film-with-g30-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sweedr-model-300-strap-chopper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-8-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/transparent-white-c-a-film-with-g30-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-5-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-26-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-6-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:33 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 12:50:33 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/transparent-white-c-a-film-with-g30-12-4-mil.html>: HTTP status code is not handled or not allowed 2025-11-08 12:50:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-6-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-6-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-10-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-46-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/56-x-60-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-40-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-42-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-x-30-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-3-x-15case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/6-x-10-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 12:50:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-30-black-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-friction-reduction-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-inch-x-18-yard-heat-sealing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tensilized-strapping-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-3m-371-plus-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-in-x-60-yd-filament-tape-rg15.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-42-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-9100-extra-heavy-duty-clear-machine-length-packaging-tape-3-x-1000-yards-2-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-bags-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/80mm-heavy-duty-tape-dispenser-for-foam-tapes.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-7151qt-medium-duty-cold-temp-clear-machine-length-packaging-tape-2-x-1000-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fromm-p328-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-36-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ld1-belt-set.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b2000-battery-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-4-x-20-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b1000-strapping-tool-feed-wheel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-x-30-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-spkt-ldxrtb-61.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-shock-absorber-shk-007.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-blade-guard-w-sponge-fj-01-06.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg20-machine-length-tape-2-x-1000.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg20-machine-length-tape-3-x-1000.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-1425-standard-duty-steel-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-ms20-spare-parts-kit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-mbd22-spare-parts-kit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-arx-uniform-case-indexer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-12-x-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-palletpal-360-air-level-loader.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-e-z-reach-portable-container-tilter-2000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-24-x-36-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/9-5-x-9-875-curby-mailer.html returned 404 status code. 2025-11-08 12:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-1000be-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/shurtape-hp200-3-inch-x-110-yd-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-industrial-warehouse-trailer-48x96.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-masterplat-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-5000-zenith-mid-machine-film-63-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/stretch-wrap-optimization.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b300-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/service-request.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b300-battery-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b400-strapping-tool-feed-wheel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-spkt-xrtb-nt-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-12-x-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:58 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/contact-us) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/contact-us landed on page that is not a product page. 2025-11-08 12:50:58 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/stretch-wrap-optimization.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/stretch-wrap-optimization.html landed on page that is not a product page. 2025-11-08 12:50:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-industrial-declares-death-to-packing-peanuts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:50:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/service-request.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/service-request.html landed on page that is not a product page. 2025-11-08 12:50:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-aaron-stelzl returned 404 status code. 2025-11-08 12:51:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-028-x-6500-green-polyester-embossed-tool-grade-strapping-coil-785-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-industrial-named-a-great-place-to-work-for-the-fourth-year-in-a-row already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-industrial-declares-death-to-packing-peanuts) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-industrial-declares-death-to-packing-peanuts landed on page that is not a product page. 2025-11-08 12:51:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-1500-amtopp-hand-stretch-film-x-treme.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/how-to-choose-the-right-type-of-safety-gloves returned 404 status code. 2025-11-08 12:51:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/breaking-up-with-plastic-sustainable-packaging-alternatives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/general-purpose-poly-strapping-kit.html returned 404 status code. 2025-11-08 12:51:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-scr-drive-board-e137-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-12-225-edge-protector.html returned 404 status code. 2025-11-08 12:51:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-72-120-edge-protector.html returned 404 status code. 2025-11-08 12:51:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-60-25-edge-protectors.html returned 404 status code. 2025-11-08 12:51:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1000-hand-stretch-film-zenith-120-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:02 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-industrial-named-a-great-place-to-work-for-the-fourth-year-in-a-row) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-industrial-named-a-great-place-to-work-for-the-fourth-year-in-a-row landed on page that is not a product page. 2025-11-08 12:51:02 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/breaking-up-with-plastic-sustainable-packaging-alternatives) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/breaking-up-with-plastic-sustainable-packaging-alternatives landed on page that is not a product page. 2025-11-08 12:51:02 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/15-inch-hand-stretch-wrap.html returned 404 status code. 2025-11-08 12:51:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fold-1-ply-paper-15-x-1650.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fold-1-ply-paper-15-x-1188.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/biodegradable-packing-peanuts.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-1000b-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cf-5-box-folding-pack-station.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/standard-duty-strapping-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1000-hand-stretch-film-zenith-120-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-7000r-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-nexus-machine-film-115-gauge-stretch-wrap-4000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-slider-top-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/anser-u2-pro-s-mounted-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polypropylene-polyester-side-serratable-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/orgapack-200-battery.html returned 404 status code. 2025-11-08 12:51:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/flexible-flat-lacing-rod-for-feeding-strap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bi-directional-poly-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/premium-front-action-serratable-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/basic-starter-spare-parts-kit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-stretch-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-6-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/front-action-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 1829 pages (at 365 pages/min), scraped 938 items (at 163 items/min) 2025-11-08 12:51:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-54-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-42-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-40-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-x-24-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-30-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/orange-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-inch-x-8-inch-x-24-inch-laddawn-clear-gusseted-poly-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-in-x-60-yd-filament-tape-rg300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vibac-5000-series-machine-carton-sealing-tape-2-1000.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-in-x-60-yd-filament-tape-rg303.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-4-x-15-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-42-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-in-bar-print-packing-list-enclosed-envelope.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-3-inch-tape-head-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/start-plugin-electric-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-in-x-60-yd-filament-tape-rg303.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eastey-em1622t-performance-series-manual-hot-wire-l-bar-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-ply-rough-top-belt-set.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ld3sb-top-squeezers-tsa3sb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/little-david-cf-50t-automatic-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-10-x-30-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/seal-a-tron-food-grade-stainless-steel-l-bar-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-6500-tl-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/polychem-reg-b800-battery-powered-strapping-tool.html returned 404 status code. 2025-11-08 12:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/culinary-basics-hinged-9-5-x-10-5.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-9-x-32-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-36-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-loveshaw-tape-blade-4m2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-white-vinyl-tape-1-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sharp-max-12-roll-bagging-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3x600-natural-water-activated-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-tartan-305-2-inch-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:36 [py.warnings] (PID: 112) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-08 12:51:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-mh400-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fromm-fr2000-robotic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-rpkt-cac60hs20.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:37 [scrapy.extensions.feedexport] (PID: 112) ERROR: Big Query insert errors (1): [{'index': 485, 'errors': [{'reason': 'invalid', 'location': 'prices[0].quantity', 'debugInfo': '', 'message': 'Cannot convert value to integer (bad value): None'}]}] 2025-11-08 12:51:37 [scrapy.extensions.feedexport] (PID: 112) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:1 2025-11-08 12:51:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-01-2460-standard-plastic-plastic-twist-tie-ribbon.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:37 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ideas/ebooks/strapping-banding-ebook.html returned 404 status code. 2025-11-08 12:51:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/madison-tm-21a-automatic-orbital-tape-bander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-yellow-vinyl-tape-2-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:38 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/packaging-testing.html returned 404 status code. 2025-11-08 12:51:38 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/engineering-success-developing-future-packaging returned 404 status code. 2025-11-08 12:51:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/anser-u2-smart-mounted-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:38 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/war-on-waste-what-you-need-to-know returned 404 status code. 2025-11-08 12:51:38 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/small-footpring-big-savings-from-a-palletizer returned 404 status code. 2025-11-08 12:51:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-harder-united-merger already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/packaging-fails-cost-of-shipping-damage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/breaking-down-blockchain-part-1 returned 404 status code. 2025-11-08 12:51:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/great-place-to-work-fifth-consecutive-year already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/key-factors-to-consider-when-buying-strapping-equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-3-white-perforated-thermal-transfer-labels-2000-labels.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-white-perforated-thermal-transfer-labels-1000-labels.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-2-inch-ac-tape-heads.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/pack-to-the-future-packaging-predictions returned 404 status code. 2025-11-08 12:51:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b800-replacement-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-2-white-non-perforated-thermal-transfer-labels-2900-labels.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/packaging-fails-cost-of-shipping-damage) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/packaging-fails-cost-of-shipping-damage landed on page that is not a product page. 2025-11-08 12:51:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-harder-united-merger) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-harder-united-merger landed on page that is not a product page. 2025-11-08 12:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-heartland-white-perforated-transfer-labels.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/amazons-new-machines-pack-five-times-faster-than-humans returned 404 status code. 2025-11-08 12:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-3-120-strapping-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/great-place-to-work-fifth-consecutive-year) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/great-place-to-work-fifth-consecutive-year landed on page that is not a product page. 2025-11-08 12:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-3-225-strapping-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/key-factors-to-consider-when-buying-strapping-equipment) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/key-factors-to-consider-when-buying-strapping-equipment landed on page that is not a product page. 2025-11-08 12:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b600-b1200-battery-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-3-225-strapping-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-3-x-2-160-strapping-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-industrial-acquires-preferred-tape-inc already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-reset-switch.html returned 404 status code. 2025-11-08 12:51:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-4-x-3-120-strapping-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-30-120-edge-protector.html returned 404 status code. 2025-11-08 12:51:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-airspace-g4-air-pillow-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ep-820-ball-knob-dispenser.html returned 404 status code. 2025-11-08 12:51:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ipg-stretchflex-sf1-100-gauge-stretch-wrap.html returned 404 status code. 2025-11-08 12:51:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-industrial-acquires-preferred-tape-inc) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-industrial-acquires-preferred-tape-inc landed on page that is not a product page. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-emitters-diffusers/zerust-vc2-2-vci-vapor-capsules.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-emitters-diffusers/zerust-vc1-1-vci-vapor-capsules.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-lp-sw-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handi-foil-full-size-heavy-duty-recyclable-aluminum-pan.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-emitters-diffusers/zerust-vc2-1-vci-vapor-capsules.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-inch-trusted-guard-freezer-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nestaflex-226-flexible-gravity-conveyor.html#579=9471 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/resealable-xl-white-block-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-16-x-24-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/resealable-xl-white-block-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fold-1-ply-paper-30-x-1188.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nestaflex-226-flexible-gravity-conveyor.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-slider-top-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-8-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-54-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-8-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-30-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-48-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/56-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-12-x-36-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-60-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-16-x-40case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-3-x-15-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:51:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-24-x-52case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-12-x-30case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-24-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1000-yd-packing-tape-clear.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-14-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-24case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/green-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-60-yd-intertape-534-flatback-paper-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1000-yd-packing-tape-clear.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-60-yds-duct-tape-20.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-tape-cartridge-blade-loveshaw.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-24-x-52case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-endless-belt-set-for-ld16sb-tapers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-microjet-hrp-cts-3-controller.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-venom-light-duty-reinforced-water-activated-gummed-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/clear-face-document-envelopes-pl70.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-replacement-case-sealer-drive-belt.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/start-battery-powered-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-duct-tape-6.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/platform-extension-ld3sb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-pt7-uv-resistant-painters-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 2127 pages (at 298 pages/min), scraped 1088 items (at 150 items/min) 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/water-tight-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychemr-b600-battery-powered-handheld-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/clear-face-document-envelopes-pl70.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-110-yd-cold-temp-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/poly-pouch-seal-maker.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-15-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/airmove-wrap-cushioning-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-traveler-mobile-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/astro-dg2-manual-handgun-hmt-compatible-dg401-rs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-wheel-fjg-1a-149.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-torsion-spring-fj-h-01.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-rpk-7-cac60.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-15-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/clear-face-document-envelopes-pl70.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:09 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/good-natured-70-gauge-machine-stretch-wrap.html returned 404 status code. 2025-11-08 12:52:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/toptier-conventional-high-infeed-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rollbag-r1275-automatic-bagger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-compacta-tire-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:10 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/eagle-q31-battery-powered-strapping-tool.html returned 404 status code. 2025-11-08 12:52:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/columbia-machine-fl3000-floor-level-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-100-black-poly-sheeting-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/about-us.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/specialty-custom-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-bronze-flange-bushing-50186-007.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/product-launch-packaging-ebike.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-to-prevent-load-failure-damage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-2-inch-top-tension-roller-78-8052-6565-5.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-1-4-inch-gatorstrap-phosphate-wire-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-gatorstrap-phosphate-wire-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas landed on page that is not a product page. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-gatorstrap-phosphate-wire-buckles-500-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/about-us.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/about-us.html landed on page that is not a product page. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-shrinkable-band.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/perforated-heat-shrinking-bands.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-1-4-inch-phosphate-coated-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/specialty-custom-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/specialty-custom-packaging.html landed on page that is not a product page. 2025-11-08 12:52:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-phosphate-coated-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stretch-wrapping-solutions.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-to-prevent-load-failure-damage) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-to-prevent-load-failure-damage landed on page that is not a product page. 2025-11-08 12:52:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/product-launch-packaging-ebike.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/product-launch-packaging-ebike.html landed on page that is not a product page. 2025-11-08 12:52:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-hd-wire-strapping-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-wire-strapping-buckles.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/everything-you-need-to-know-about-water-activated-tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/atp-cvt-636-colored-vinyl-tape-logs.html returned 404 status code. 2025-11-08 12:52:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/meet-inside-sales-representative returned 404 status code. 2025-11-08 12:52:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/hello-downtown-rocket-industrials-new-digs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/stretch-wrapping-solutions.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/stretch-wrapping-solutions.html landed on page that is not a product page. 2025-11-08 12:52:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/navigate-supply-shortages-for-packaging-materials returned 404 status code. 2025-11-08 12:52:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-x-020-standard-grade-steel-strapping-1800-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-x-020-standard-grade-steel-strapping-1500-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/everything-you-need-to-know-about-water-activated-tape) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/everything-you-need-to-know-about-water-activated-tape landed on page that is not a product page. 2025-11-08 12:52:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-020-standard-grade-steel-strapping-1200-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-60-120-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7th-bbl-paper-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-brown-paper-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ribbon-wound-strapping-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/samuel-welding-clamp.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/hello-downtown-rocket-industrials-new-digs) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/hello-downtown-rocket-industrials-new-digs landed on page that is not a product page. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/quart-brown-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/kubinec-tension-bar.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6th-bbl-75-paper-grocery-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/torque-70-hand-pallet-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6th-barrel-65-paper-grocery-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-60-120-edge-protector.html returned 404 status code. 2025-11-08 12:52:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-36-160-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-50-gauge.html returned 404 status code. 2025-11-08 12:52:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-oz-family-size-deli-container.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-24-static-control-bubble-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handi-foil-8-inch-recyclable-square-aluminum-cake-pan.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-lb-red-plaid-paper-food-tray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-75-gauge.html returned 404 status code. 2025-11-08 12:52:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-clear-resealable-bag-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/semi-auto-stretch-wrapper-high-profile.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-flat-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-rotoplat-708-semi-auto-stainless-steel-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-inch-reclosable-bags-2-mil0.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-36-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/lx-500p-compact-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-oz-family-size-deli-container.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microraves-12-oz-microwaveable-combo-pack-food-containers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-28-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-5-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-36-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/34-x-40-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/44-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-high-temp-masking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-42-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-60-yd-filament-tape-rg15.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-110-yd-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-513-light-industrial-masking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-appliance-furniture-securing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/52-x-60-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-stop-caution-233-reinforced-printed-water-activated-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/6-x-12-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 12:52:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/quad-725-general-purpose-hot-melt-glue-stick.html returned 404 status code. 2025-11-08 12:52:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3m-deluxe-packing-tape-dispenser.html returned 404 status code. 2025-11-08 12:52:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-brake-washer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/crumple-kraft-paper-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/flush-mount-turntable-lift.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/premium-cord-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/adhesives-glue-systems/glue-guns.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-limit-switch-nlca2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-evolution-lt-battery-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/airmove2-air-pillow-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/colored-marking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-coarse-tooth-loveshaw-tape-blade-m2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-450ht-adjustable-temp-hot-melt-glue-gun-5-8-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/western-adhesives-hot-melt-glue-gun-1-2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fromm-p329-s-friction-weld-banding-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-10-x-36-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-zippered-poly-bags-10-x-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-10-x-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/tach-it-3560-semi-automatic-twist-tie-machine.html returned 404 status code. 2025-11-08 12:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/yellow-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-zls-low-profile-lift-table-2000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-5-in-x-60-yd-blue-painters-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vc2-1-vci-vapor-capsules.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/astro-ss10-hot-melt-tank.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/automotive-transportation-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bundle-and-save.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/alternative-materials.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:50 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-75-x-35-mailing-tubes.html returned 404 status code. 2025-11-08 12:52:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/astro-ss10-hot-melt-tank.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/refresh-azure-foam-soap.html returned 404 status code. 2025-11-08 12:52:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/beverage-distribution.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/animals-in-packaged-food returned 404 status code. 2025-11-08 12:52:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/automotive-transportation-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/automotive-transportation-packaging.html landed on page that is not a product page. 2025-11-08 12:52:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/driving-dramatic-growth-robotics returned 404 status code. 2025-11-08 12:52:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/bundle-and-save.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/bundle-and-save.html landed on page that is not a product page. 2025-11-08 12:52:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/packaging-automation-101-where-to-begin returned 404 status code. 2025-11-08 12:52:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/industries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/alternative-materials.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/alternative-materials.html landed on page that is not a product page. 2025-11-08 12:52:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/were-keeping-an-eye-on-these-food-packaging-trends already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:53 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/will-germanys-new-packaging-law-impact-you returned 404 status code. 2025-11-08 12:52:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/polyair-x-fold-1-ply-paper-30-x-1188.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-knife-guard-assembly-kga60-v.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/polyair-x-fold-1-ply-paper-30-x-990.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/packaging-plays-vital-role-covid-19-vaccine returned 404 status code. 2025-11-08 12:52:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/beverage-distribution.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/beverage-distribution.html landed on page that is not a product page. 2025-11-08 12:52:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/polyair-x-fold-2-ply-paper-30-x-1155.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/polyair-x-fold-2-ply-paper-30-x-990.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/polyair-x-fold-1-ply-paper-15-x-1650.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:55 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/category/industries) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/category/industries landed on page that is not a product page. 2025-11-08 12:52:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/polyair-x-fold-1-ply-paper-15-x-1188.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/18-x-18-kraft-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-10-x-12-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/food-processor-silicone-spray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/black-gaffers-tape-low-gloss-finish.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/gray-gaffers-tape-low-gloss-finish.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-1-4-inch-x-600-woven-poly-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-30-160-edge-protector.html returned 404 status code. 2025-11-08 12:52:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-kraft-paper-sheets-40-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/were-keeping-an-eye-on-these-food-packaging-trends already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:52:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/nwd-littlenelson.html returned 404 status code. 2025-11-08 12:52:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-sheets/18-x-18-kraft-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stretch-wrapper-photoeye.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/strain-relief-connector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/were-keeping-an-eye-on-these-food-packaging-trends) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/were-keeping-an-eye-on-these-food-packaging-trends landed on page that is not a product page. 2025-11-08 12:53:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-nexus-machine-film-45-gauge-stretch-wrap-10000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-high-temp-masking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-global-force-machine-film-65-gauge-stretch-wrap-7000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-poly-tubing-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-inch-small-hand-wrapping-film-120-ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/50ft-half-hard-wire-18ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/12-inch-stretch-film.html returned 404 status code. 2025-11-08 12:53:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/scoring-blade-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/automatic-high-profile-pallet-wrapping-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/resealable-white-block-parts-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/tape/tape-dispensers-applicators/desktop-tape-dispensers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-lb-red-plaid-food-tray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dyne-a-pak-heavy-duty-black-foam-meat-tray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:06 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/resealable-static-free-bag.html returned 404 status code. 2025-11-08 12:53:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/150mm-electronic-heavy-duty-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/carousel-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-wide-auto-feed-cut-tape-dispenser-with-safety-guard.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/start-digital-automatic-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/airmover-replacement-bubble-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/double-sided-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-sl3-manual-definite-length-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 2457 pages (at 330 pages/min), scraped 1239 items (at 151 items/min) 2025-11-08 12:53:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-inch-black-conductive-tubing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-inch-clear-poly-tubing-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-x-26-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-10-x-36-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-4-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-inch-torque-ii-hand-stretch-film-1500-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-8-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-48-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/poly-barrel-cover.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-15-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-12-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-4-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-7-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-4-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-48-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-7-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-36-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-4-x-20-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-24-x-60-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1000-yard-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-duct-tape-30.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-1500-yard-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-10-x-48case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vibac-730-2-inch-x-110-yd-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wide-pack-table-for-3-inch-ld7.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-rubber-wipe-roller-assembly.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-11-1-2-self-seal-bubble-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-40-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-fast-set-hot-melt-glue-stick-1-2-inch-10-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:28 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/18-x-23-1-2-self-seal-bubble-pouches.html returned 404 status code. 2025-11-08 12:53:29 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/hand-saver-dispenser-w-tape.html returned 404 status code. 2025-11-08 12:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-evolution-lt-replacement-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-9-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:30 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/good-natured-80-gauge-hand-stretch-wrap.html returned 404 status code. 2025-11-08 12:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t20cf-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-c-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-vacuum-cup-vc-1001.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t20cf-sm-small-box-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-endless-belt-set-95-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-9-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/premium-heat-shrink-gun.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-22-x-16-x-30-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg20-packaging-tape-2-x-110.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/streamfeeder-st-1250-friction-feeder-demo.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handle-it-model-3000-mobile-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/premium-heat-shrink-gun.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-portable-dandy-lift-table-500-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-genesis-futura-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/streamfeeder-st-1250-friction-feeder-demo.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-mt300-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30lb-kraft-paper-roll-36x1200.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/madison-mpt-17-orbital-stretch-film-bander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:33 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/scapa-136-white-polyethylene-film-tape-50-x-60.html returned 404 status code. 2025-11-08 12:53:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/shrink-wrapper-and-heat-tunnel-combo.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/services/rocket-stock-it.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tape-slitting-and-converting.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eastey-etb-series-shrink-bundling-tunnel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wausau.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ld3sb-hand-knob-psu166-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/why-is-a-pizza-box-square already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-1200b-hp-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/tape-slitting-and-converting.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/tape-slitting-and-converting.html landed on page that is not a product page. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/services/rocket-stock-it.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/ultimate-guide-to-choosing-the-best-box-sealing-tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/2023-year-in-review already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/this-month-in-packaging-march-2021 returned 404 status code. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-flush-cut-bubble-pouches-bob68f.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-17-1-2-self-seal-bubble-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-1-2-self-seal-bubble-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/wausau.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/wausau.html landed on page that is not a product page. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-resealable-bubble-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/why-is-a-pizza-box-square) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/why-is-a-pizza-box-square landed on page that is not a product page. 2025-11-08 12:53:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/services/rocket-stock-it.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/services/rocket-stock-it.html landed on page that is not a product page. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-028-x-3250-black-polyester-smooth-tool-grade-strapping-820-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-resealable-bubble-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/ultimate-guide-to-choosing-the-best-box-sealing-tape) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/ultimate-guide-to-choosing-the-best-box-sealing-tape landed on page that is not a product page. 2025-11-08 12:53:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/2023-year-in-review) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/2023-year-in-review landed on page that is not a product page. 2025-11-08 12:53:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-020-x-7200-green-polyester-smooth-machine-grade-strapping-600-lb-16-x-6-core.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/history-of-the-n95-mask returned 404 status code. 2025-11-08 12:53:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-sm100-tow-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/automation-road-map returned 404 status code. 2025-11-08 12:53:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cord-wall-mount-strapping-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ip-321-red-machine-tape-48mmx914m.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-40-120-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-wrapper-sprocket.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fill-pro-paper-void-fill-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-48-120-edge-protector.html returned 404 status code. 2025-11-08 12:53:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-cleany-plastic-blade-scraper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-january already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-16-x-24-static-control-bubble-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/safety-scraper-blades.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/motor-control-card.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/superflex-42-gauge.html returned 404 status code. 2025-11-08 12:53:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000f-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-1-2-x-14-1-2-poly-bubble-mailers-b834.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/8-x-22-shrink-bag.html returned 404 status code. 2025-11-08 12:53:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/safety-cutter-replacement-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-january) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-january landed on page that is not a product page. 2025-11-08 12:53:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-8-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-16-x-24-static-control-bubble-packaging.html returned 404 status code. 2025-11-08 12:53:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stretch-wrapper-switch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-14-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-48-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-28-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-18-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/38-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-54-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:53:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-24-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/34-x-40-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-14-x-36-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/plastic-lid-8oz-cup.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/56-x-60-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-24-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/34-x-40-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-8-x-20-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-12-x-30-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:03 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-inch-x-1500-yard-packing-tape.html returned 404 status code. 2025-11-08 12:54:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yd-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-7151qt-medium-duty-cold-temp-clear-machine-length-packaging-tape-3-x-1000-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-hot-melt-glue-sticks-cold-resistant-1-2-inch-10-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-pvc-aisle-marking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-made-in-the-usa-240-reinforced-printed-water-activated-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-20-x-48-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-finger-plate-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-7100-medium-duty-tan-packaging-tape-2-x-110-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-venom-light-duty-reinforced-water-activated-gummed-tape-450-feet.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-bushing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1500-yard-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-pvc-aisle-marking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/high-speed-kit-for-ld7-ldx-crs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-20-ungummed-kraft-jumbo-envelopes.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-drive-belt-27100331.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-515-general-purpose-hd-masking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/toptier-robotic-pick-and-place-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-technoplat-708-cs-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-manual-pallet-turntable-6000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-s867-l-clip-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sharp-max-pro-18-roll-bagging-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-100-clear-poly-sheeting-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 2784 pages (at 327 pages/min), scraped 1394 items (at 155 items/min) 2025-11-08 12:54:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-blue-vinyl-tape-1-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/harder-united.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/explore-packaging-equipment.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/million-pound-promise.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-paper-interleaving-products.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rollbag-r3200xl-high-speed-automatic-bagger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-drive-belt-27100331.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/3-packaging-tips-damage-prevention already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/7-point-checklist-maintaining-tape-head already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/5-steps-to-make-your-packaging-more-sustainable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/were-in-this-together-rockets-covid19-statement returned 404 status code. 2025-11-08 12:54:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg20-packaging-tape-3-x-110.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/million-pound-promise.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/million-pound-promise.html landed on page that is not a product page. 2025-11-08 12:54:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/recycling-banding-materials-with-a-strap-chopper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/harder-united.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/explore-packaging-equipment.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/explore-packaging-equipment.html landed on page that is not a product page. 2025-11-08 12:54:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/5-advantages-of-working-with-a-packaging-distributor already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/sustainability-in-the-ecommerce-world returned 404 status code. 2025-11-08 12:54:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/3-packaging-tips-damage-prevention) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/3-packaging-tips-damage-prevention landed on page that is not a product page. 2025-11-08 12:54:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/5-steps-to-make-your-packaging-more-sustainable) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/5-steps-to-make-your-packaging-more-sustainable landed on page that is not a product page. 2025-11-08 12:54:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-paper-interleaving-products.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-to-choose-your-beer-packaging-partner already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/recycling-banding-materials-with-a-strap-chopper) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/recycling-banding-materials-with-a-strap-chopper landed on page that is not a product page. 2025-11-08 12:54:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/harder-united.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/harder-united.html landed on page that is not a product page. 2025-11-08 12:54:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/7-point-checklist-maintaining-tape-head) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/7-point-checklist-maintaining-tape-head landed on page that is not a product page. 2025-11-08 12:54:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/5-advantages-of-working-with-a-packaging-distributor) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/5-advantages-of-working-with-a-packaging-distributor landed on page that is not a product page. 2025-11-08 12:54:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/the-history-of-cheese-packaging already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-77-spray-adhesive.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/stretch-wrap-vs-shrink-wrap already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/rocket-paper-interleaving-products.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/rocket-paper-interleaving-products.html landed on page that is not a product page. 2025-11-08 12:54:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/should-i-consider-a-used-stretch-wrap-machine returned 404 status code. 2025-11-08 12:54:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/remote-monitoring-becoming-essential-for-many-operations returned 404 status code. 2025-11-08 12:54:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-to-choose-your-beer-packaging-partner) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-to-choose-your-beer-packaging-partner landed on page that is not a product page. 2025-11-08 12:54:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/the-history-of-cheese-packaging) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/the-history-of-cheese-packaging landed on page that is not a product page. 2025-11-08 12:54:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/reduce-dimensional-weight-shipping-costs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-40-160-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-30-160-edge-protector.html returned 404 status code. 2025-11-08 12:54:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/17-5-inch-spartan-47-gauge-hand-stretch-film-1500-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-6-160-strapping-protectors.html returned 404 status code. 2025-11-08 12:54:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/20-inch-extended-handle-stretch-wrap.html returned 404 status code. 2025-11-08 12:54:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/maxisafe-deep-cut-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-industrial-earns-2021-great-place-to-work-certification already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/charger-for-orgapack-250-400-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/stretch-wrap-vs-shrink-wrap) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/stretch-wrap-vs-shrink-wrap landed on page that is not a product page. 2025-11-08 12:54:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-feed-switch.html returned 404 status code. 2025-11-08 12:54:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-switch-automatic-a-arm-low-profile-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-1000bws-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/maxisafe-deep-cut-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/reduce-dimensional-weight-shipping-costs) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/reduce-dimensional-weight-shipping-costs landed on page that is not a product page. 2025-11-08 12:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-15-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-14-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-pre-zippered-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-vci-kraft-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/duratip-replacement-blades.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-inch-clear-poly-tubing-2-mil.html returned 404 status code. 2025-11-08 12:54:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-inch-black-conductive-tubing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/resealable-bag-imprinted-with-white-block.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-8-x-20-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-industrial-earns-2021-great-place-to-work-certification) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-industrial-earns-2021-great-place-to-work-certification landed on page that is not a product page. 2025-11-08 12:54:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t-550rl-random-size-cardboard-box-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-7-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-15-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-8-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-4-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-7-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/35lb-vci-poly-coated-kraft-paper-roll-36-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-54-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-7-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-36-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-36-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-40-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/38-x-60-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-42-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-42-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-18-x-48case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-x-48case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-4-x-24-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-green-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-56-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-24-x-60case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-8-x-30-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-16-x-40-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-guide-plate-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-56-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-safe-handling-140-printed-water-activated-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-aquamask-medium-grade-masking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-5-x-19-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/versa-pak-dispenser-box.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1640-3m-311-plus-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handle-it-1100aa-c-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-03-2500-standard-paper-plastic-twist-tie-ribbon.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-10-x-24-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-guide-roller.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b2000-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-torsion-guard-spring-spr-1063.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-pin-cac60-0042-3.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-roll-retention-key-cac60-0179-3.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/top-carton-flap-folders-ld3-ld7-ld19.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-msd22-spare-parts-kit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-main-spring-psc501101-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/replacement-parts-12-inch-seal-cutter.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/bestpack-csx-automatic-random-case-sealer.html returned 404 status code. 2025-11-08 12:54:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-standard-duty-windlass-strapping-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/madison-mpt-21-orbital-stretch-film-bander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-yellow-jacket-87-mu-orbital-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000bep-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-roll-retention-key-cac60-0179-3.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stand-encore-ep-6700-25-ring-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/manuals.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/stretch-film-analysis-paper-manufacturer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/women-in-packaging-history already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/order-information.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/usps-bulk-mailing-regulations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:54:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/right-level-of-automation returned 404 status code. 2025-11-08 12:55:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com landed on page that is not a product page. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/unpacking-the-plastic-waste-problem already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/kubinec-3-4-1650-woven-poly-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-x-1650-woven-poly-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-inch-x-2000-woven-poly-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/greener-way-to-protect-products-during-shipment returned 404 status code. 2025-11-08 12:55:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/manuals.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/manuals.html landed on page that is not a product page. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/gatorlash-1-5-8-x-670-heavy-duty-poly-cord-lashing-11000-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/gatorstrap-1-1-4-x-600-heavy-duty-poly-cord-strapping-3285-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zippstrap-3-8-x-5250-high-strength-bonded-poly-cord-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/order-information.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/order-information.html landed on page that is not a product page. 2025-11-08 12:55:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-brian-garvin returned 404 status code. 2025-11-08 12:55:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/usps-bulk-mailing-regulations) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/usps-bulk-mailing-regulations landed on page that is not a product page. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-x-4000-woven-poly-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/gatorstrap-1-x-2500-hd-poly-cord-strapping-3100-lb.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/services already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-bags.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-25-x-19-x-39-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-40-x-36-x-80-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/unpacking-the-plastic-waste-problem) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/unpacking-the-plastic-waste-problem landed on page that is not a product page. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-15-x-9-x-24-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-18-x-24-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/women-in-packaging-history) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/women-in-packaging-history landed on page that is not a product page. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-6-x-8-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-flat-poly-bags-4-x-6-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-inch-x-2000-woven-poly-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/battery-tool-grade-green-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:02 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/stretch-film-analysis-paper-manufacturer.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/stretch-film-analysis-paper-manufacturer.html landed on page that is not a product page. 2025-11-08 12:55:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nifty-2-inch-handheld-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nifty-3-inch-heavy-duty-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tape-logic-3-inch-standard-industrial-tape-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:03 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/services) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/services landed on page that is not a product page. 2025-11-08 12:55:03 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-72-160-edge-protector.html returned 404 status code. 2025-11-08 12:55:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-36-160-edge-protector.html returned 404 status code. 2025-11-08 12:55:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-intertape-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 3076 pages (at 292 pages/min), scraped 1524 items (at 130 items/min) 2025-11-08 12:55:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-nexus-machine-film-70-gauge-stretch-wrap-6500-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:09 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-100-gauge.html returned 404 status code. 2025-11-08 12:55:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-nexus-machine-film-55-gauge-stretch-wrap-6000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/secumax-350.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/astro-8-foot-heated-hose-hmt-compatible-dg0608-r.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-intertape-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-34-x-31-x-69-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-infeed-pack-and-exit-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40lb-kraft-paper-roll-24x900.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/predator-low-profile-turntable.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/aluminum-stretch-wrap-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/12-x-22-vacuum-pouch.html returned 404 status code. 2025-11-08 12:55:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-flat-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/anti-static-esd/static-shielding-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-air-pillow-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-enter-exit-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-5-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-4-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-2-inch-reclosable-bags-2-mil4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-48-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-14-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-4-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-10-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-7-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-28-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-16-x-40-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-12-x-30-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-8-x-24-gusseted-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1500-yd-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/white-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-clear-portion-cups-epc200-case-of-2500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-appliance-furniture-securing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-110-yd-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/connection-infeed-kit-table-cf-5.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-20-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-12-x-36case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/44-x-60-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-5-x-12-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-seal-bag-closer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:45 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3m-lane-marking-applicator.html returned 404 status code. 2025-11-08 12:55:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/astro-d2-15-hot-melt-tank.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-replacement-upper-tape-head-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-rsa-2625-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-replacement-lower-tape-head-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-idler-wheel-assembly-fg-01a-20assy.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-450-carton-master-reinforced-water-activated-gummed-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t400r-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-rpk-7-cac61.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-in-x-60-yd-filament-tape-rg15.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-compression-spring-spr-1045.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ls-plus-scissors-lift.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fromm-p328-s-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-blue-vinyl-tape-2-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-order-picking-platform-cart-42x48.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-plc-k7m-dr14ue.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/atp-pgm-uv14-blue-painters-masking-tape-39-x-60.html returned 404 status code. 2025-11-08 12:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t400r-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-portable-dandy-lift-table-330-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-36-powered-turntable-4000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-compression-spring-spr-1045.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-pre-stretch-motor-nmrv040-10-ys634.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-inch-x-3-inch-x-15-inch-laddawn-clear-gusseted-poly-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-rpk-7-cac61.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/stretch-wrapper-implementation.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/industrial-manufacturing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/about-the-company/sustainability.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/dedicated-to-your-success already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/alpha-compact-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/create-your-own-job.html returned 404 status code. 2025-11-08 12:55:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/policies.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/industrial-manufacturing.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/industrial-manufacturing.html landed on page that is not a product page. 2025-11-08 12:55:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/stretch-wrapper-implementation.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/stretch-wrapper-implementation.html landed on page that is not a product page. 2025-11-08 12:55:52 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 12:55:52 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/loveshaw-spare-parts-kit-rpk-7-cac61.html>: HTTP status code is not handled or not allowed 2025-11-08 12:55:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-july already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/packaging-for-small-businesses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/boost-productivity-cut-costs-with-the-robopac-s7 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/factors-to-consider-before-automating returned 404 status code. 2025-11-08 12:55:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/wrap-rage-symptoms-causes-treatment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/now-open-packlytics-packaging-test-lab returned 404 status code. 2025-11-08 12:55:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microjet-plus-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microjet-ii-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microjet-iii-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/matthews-v-series-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:53 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-july) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-july landed on page that is not a product page. 2025-11-08 12:55:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-high-resolution-printer-hrp.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:53 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/policies.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/policies.html landed on page that is not a product page. 2025-11-08 12:55:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/matthews-l-series-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/matthews-mperia-universal-printer-controller.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:53 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/dedicated-to-your-success) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/dedicated-to-your-success landed on page that is not a product page. 2025-11-08 12:55:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/about-the-company/sustainability.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/packaging-for-small-businesses) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/packaging-for-small-businesses landed on page that is not a product page. 2025-11-08 12:55:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ct1000-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/evolution-1-high-resolution-industrial-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/boost-productivity-cut-costs-with-the-robopac-s7) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/boost-productivity-cut-costs-with-the-robopac-s7 landed on page that is not a product page. 2025-11-08 12:55:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-robotic-stretch-wrapper-86-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-robot-master-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/motix-1h-handheld-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/wrap-rage-symptoms-causes-treatment) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/wrap-rage-symptoms-causes-treatment landed on page that is not a product page. 2025-11-08 12:55:55 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-holding-gripper.html returned 404 status code. 2025-11-08 12:55:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/repair-or-replace-how-to-assess-your-packaging-equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/about-the-company/sustainability.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/about-the-company/sustainability.html landed on page that is not a product page. 2025-11-08 12:55:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-24-120-edge-protector.html returned 404 status code. 2025-11-08 12:55:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-12-160-edge-protector.html returned 404 status code. 2025-11-08 12:55:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-airspace-g6-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/repair-or-replace-how-to-assess-your-packaging-equipment) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/repair-or-replace-how-to-assess-your-packaging-equipment landed on page that is not a product page. 2025-11-08 12:55:59 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/12-inch-clear-hand-wrap.html returned 404 status code. 2025-11-08 12:55:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stainless-steel-allfit-trapezoid-blades.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:55:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-secunorm-380-semi-automatic-retractable-spring-loaded-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-airspace-air-pillow-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/torque-i-hand-stretch-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/packlytics-stretch-wrap-monitoring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fold-1-ply-paper-30-x-990.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/take-a-label-tal-450-electric-photo-eye-label-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-trusted-guard-freezer-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-resealable-bubble-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/automatic-case-folder-and-bottom-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-pre-opened-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-standard-blade-92.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-trusted-guard-freezer-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-flat-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 3343 pages (at 267 pages/min), scraped 1638 items (at 114 items/min) 2025-11-08 12:56:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-28-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-3-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:10 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/10-x-22-flairpak-400.html returned 404 status code. 2025-11-08 12:56:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-30-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-28-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-32-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-7-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-32-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-26-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-26-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/34-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-x-48-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-10-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/purple-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1000-yard-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/38-x-42-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-24-x-60case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-3-tape-head-blade-10122-0.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-1100-extra-heavy-duty-clear-machine-length-packaging-tape-2-x-1000-yards-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/replacement-drive-belt-3m-case-sealers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-60-yd-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-fragile-handle-with-care-260-reinforced-printed-water-activated-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b800-strapping-tool-feed-wheel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b2000-handheld-battery-powered-strapping-tool-batteries-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-hp-3200-high-profile-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-3510a-handheld-twist-tie-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-photo-eye.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-roller-cac60-0002-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ptfe-coated-roller.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/upkeep-kit-18-inch-water-tight-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:38 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ipg-pg49-paper-masking-tape-59-x-60.html returned 404 status code. 2025-11-08 12:56:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/texwrapr-st-2219-automatic-in-line-l-bar-sealer-19-x-15-5-x-7-max-package-size.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eastey-em1622t-hot-wire-l-bar-sealer-used.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-case-sealer-belt-ldu-1128a-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100r-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/terms.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/chipboard-sheet-17-125-x-24-75.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/preventative-maintenance-saves-time-money already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/idaho.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-tow300-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/steel-vs-polyester-strapping-infographic already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/social-distancing-in-manufacturing-will-spike-trend-adoption returned 404 status code. 2025-11-08 12:56:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-manual-portable-dandy-lift-table-330-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/terms.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/terms.html landed on page that is not a product page. 2025-11-08 12:56:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-100-clear-poly-sheeting-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/warehouse-safety-tips-ensuring-a-secure-working-environment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/idaho.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/chipboard-sheet-17-125-x-24-75.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/what-is-sustainable-packaging-design-examples already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/mile-of-music-mike-maimone returned 404 status code. 2025-11-08 12:56:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-compacta-4-orbital-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-compacta-6-orbital-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/steel-vs-polyester-strapping-infographic) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/steel-vs-polyester-strapping-infographic landed on page that is not a product page. 2025-11-08 12:56:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-spiror-hp300-horizontal-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-yellow-jacket-87-sa-orbital-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-6700-25-ring-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/warehouse-safety-tips-ensuring-a-secure-working-environment) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/warehouse-safety-tips-ensuring-a-secure-working-environment landed on page that is not a product page. 2025-11-08 12:56:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-orbit-r5-horizontal-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/idaho.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/idaho.html landed on page that is not a product page. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-6700-25-dx-ring-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-octopus-compact-c-series-ring-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-60-yds-duct-tape-30.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:45 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/what-is-sustainable-packaging-design-examples) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/what-is-sustainable-packaging-design-examples landed on page that is not a product page. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-octopus-compact-b-series-ring-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/tape/carton-sealing-tape/hand-carton-sealing-tape.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-clear-box-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vibac-3-inch-x-110-yd-cold-temp-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/shurtape-hp200-2-inch-x-110-yd-clear-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-scotch-371-packaging-tape-2-x-110-yds-1-8-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-scotch-371-packaging-tape-3-x-110-yds-1-8-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg16-packaging-tape-3-x-110.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:46 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 12:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg16-packaging-tape-2-x-110.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:46 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/chipboard-sheet-17-125-x-24-75.html>: HTTP status code is not handled or not allowed 2025-11-08 12:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-tartan-305-3-inch-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:46 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/preventative-maintenance-saves-time-money) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/preventative-maintenance-saves-time-money landed on page that is not a product page. 2025-11-08 12:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-tartan-369-2-inch-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vibac-2-inch-x-110-yd-cold-temp-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-octopus-compact-s-series-ring-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-octopus-compact-t-series-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/strapping-banding/handheld-strapping-tools/battery-powered-strapping-tools.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/expandos-standard-expander-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/vertical-kraft-paper-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b300-handheld-battery-powered-strapping-tool-batteries-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/orgapack-ort-130-handheld-banding-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/orgapack-ort-450-handheld-banding-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-b1000i-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-bxt4-19-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-bxt4-13-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-b400i-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-reg-b1200-handheld-battery-powered-strapping-tool-5-8-3-4-w-2-batteries-charger-1200-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-evolution-ht-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-siat-gt-xtreme-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-siat-viper-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-b800i-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-bxt4-16-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-spiror-hp300-horizontal-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-b800i-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-durable-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:56 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-36-160-edge-protector.html returned 404 status code. 2025-11-08 12:56:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-110-yd-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-500-medium-duty-natural-rubber-packaging-tape-2-x-110-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/extended-heavy-duty-ramp.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yd-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-lb-red-plaid-paper-food-tray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-110-yd-3m-311-plus-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-110-yd-3m-311-plus-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-s867-ii-dual-l-clip-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:56:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/shurtape-hp100-2-inch-x-110-yd-tan-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/standard-utility-knife-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-volt-bosch-battery-for-the-orgapack-ort-130-and-ort-260-banding-tools.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-narrow-hand-stretch-film-90-ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/low-profile-automatic-skid-shrink-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-12-slider-top-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/take-a-label-reg-tal-250-electric-photo-eye-label-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-slider-top-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-13-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-5-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-5-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 3588 pages (at 245 pages/min), scraped 1782 items (at 144 items/min) 2025-11-08 12:57:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-7-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-6-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-8-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-28-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-28-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/38-x-48-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-2-x-12-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-x-26-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-26-x-60-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-60-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-14-x-30-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-clear-portion-cups-epc400-case-of-2500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tabbing-metal-roll-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-60-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-x-26-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-60-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-scotch-371-packaging-tape-3-x-110-yds-1-8-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-x-48case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-8-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/pipe-bundling-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-in-red-bar-packing-list-envelope.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-3-inch-tape-head-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-inch-x-10-yd-ptfe-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-kraft-self-seal-stayflats-mailer-9-x-11-5.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/exit-extension-table-ldu-ldr.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-60-yd-masking-tape-intertape-pg29.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-40-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sweed-450-ddx-strap-chopper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/deluxe-heat-sealing-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/service-parts-16-inch-heat-seal-closer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ez-bander-5-x-1000-hand-stretch-wrap-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-upm0675-drive-belt.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/42-inch-power-lift-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-1650-pusher-rack-steel-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-spring-axis-fj-00-01.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-96-single-wall-corrugated-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-mt600-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-100-clear-poly-sheeting-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-brake-washer-psc321031-3.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-in-bar-print-packing-list-envelope.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-tamper-evident-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:32 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/robot-vs-cobot returned 404 status code. 2025-11-08 12:57:32 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/splice-up-your-life-with-our-top-selling-splicing-tapes returned 404 status code. 2025-11-08 12:57:34 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ebooks.html returned 404 status code. 2025-11-08 12:57:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-inch-reclosable-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/packaging-iq-quiz already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-may already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:34 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/new-website-launch returned 404 status code. 2025-11-08 12:57:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-to-choose-the-right-packaging-materials already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/agvs-amrs-in-the-modern-warehouse already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/stretch-wrapping/stretch-wrap/handheld-stretch-wrap.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1500-hand-stretch-film-zenith.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1500-hand-stretch-film-zenith-75-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/green-polyester-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1500-hand-stretch-film-zenith-70-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1476-hand-stretch-film-zenith-90-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/year-in-review-2021 returned 404 status code. 2025-11-08 12:57:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/hand-stretch-film-zenith.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/torque-80-hand-pallet-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/brewing-insights-tips-from-wisconsin-brewers returned 404 status code. 2025-11-08 12:57:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-may) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-may landed on page that is not a product page. 2025-11-08 12:57:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/torque-ii-hand-stretch-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1476-hand-stretch-film-zenith-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/our-response-to-current-market-challenges returned 404 status code. 2025-11-08 12:57:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/agvs-amrs-in-the-modern-warehouse) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/agvs-amrs-in-the-modern-warehouse landed on page that is not a product page. 2025-11-08 12:57:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-1500-hand-stretch-film-zenith-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/17-inch-zenith-38-gauge-prestretch-hand-stretch-film-1476-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-film-roller-assembly-fg-08-07.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-x-1500-black-stretch-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/packaging-iq-quiz) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/packaging-iq-quiz landed on page that is not a product page. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-1500-intertape-80-gauge-stretchflex-hwii-blown-hand-stretch-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-edge-90-gauge-hand-stretch-film-1500-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1500-hand-stretch-film-zenith-90-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1476-zenith-hand-stretch-film-zenith-70-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-1476-hand-stretch-film-zenith-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-inch-spartan-47-gauge-hand-stretch-film-1500-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/17-1476-hand-oriented-zenith-30-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-black-conductive-tubing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-inch-black-conductive-tubing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-inch-black-conductive-tubing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-inch-black-conductive-tubing.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-1500-hand-stretch-film-zenith.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-to-choose-the-right-packaging-materials) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-to-choose-the-right-packaging-materials landed on page that is not a product page. 2025-11-08 12:57:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-6-225-strapping-protectors.html returned 404 status code. 2025-11-08 12:57:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-30-120-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-x-75-gauge-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-inflatable-bubble-16-by-10-inch-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-hook-tip-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/60lb-vci-kraft-paper-roll-36-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-30-120-edge-protector.html returned 404 status code. 2025-11-08 12:57:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/photoeye-cable.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-2-5-lb-red-white-plaid-paper-food-trays.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-switch-high-profile-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:46 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/microjet-conditioner-fluid-640.html returned 404 status code. 2025-11-08 12:57:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/battery-for-orgapack-50-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-rotoplat-508-dw-semi-automatic-door-window-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-lb-red-plaid-food-tray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000a-stretch-wrapper-with-manual-pre-stretch-plc-and-ramp.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/staylock-clear-plastic-9-inch-medium-hinged-lid-containers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-15-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/manual-bottle-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-clear-resealable-bag-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/25ft-half-hard-wire-19ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-48-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-30-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-5-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-30-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-15-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-14-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-32-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:57:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-24-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-30-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-48-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-4-x-20-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-48-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-3-x-12-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-42-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-5-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-20-x-48-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-27-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-x-48-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-6-x-20-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-10-x-48-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-x-24-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-in-x-60-yd-flatback-paper-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/pinnaclebond-metallocene-hot-melt-adhesive-chips.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:08 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/intertape-400-medium-duty-clear-packaging-tape-2-x-110-yards.html returned 404 status code. 2025-11-08 12:58:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-18-clear-resealable-bag-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 3911 pages (at 323 pages/min), scraped 1934 items (at 152 items/min) 2025-11-08 12:58:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/greenbridge-b400i-battery-powered-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/astro-ap15-industrial-hot-melt-unit.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-ctl-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-backsaver-lite-lift-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/replacement-tape-head-for-eagle-t100-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/brown-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/airmove-void-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-eagle-replacement-tape-head-blade-w82mm.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:10 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/storopack-paperbubble.html returned 404 status code. 2025-11-08 12:58:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-film-tension-switch-z-15gq22-b.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t10cf-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/fiberglass-cloth-tape-42-x-36-yards-7-5-mil.html returned 404 status code. 2025-11-08 12:58:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-extension-spring-spr-1055.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/atp-sst-936l-black-yellow-striped-warning-tape-49-x-36.html returned 404 status code. 2025-11-08 12:58:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-cf-5af-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-case-sealer-belt-ldx-0476-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-knife-arm-spring-x111-ps.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ri-200-strapping-tool-tension-spring.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-manual-portable-dandy-lift-table-1100-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eastey-eb-professional-series-shrink-bundling-tunnel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-5-x-2-5-x-43-25-100-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/beverage-packaging-innovations-rising-trends already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-drive-roller-fjg-1a-148.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-spkt-ldx-60.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rollbag-r1285-velocity-automatic-bagger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-x-035-x-40000-green-polyester-embossed-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/loops-reusable-packaging-program-launches-in-us returned 404 status code. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-tape-core-nut-cac60-0046-3.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/guide-to-a-cleaner-healthier-facility already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/sustainability already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/beverage-packaging-innovations-rising-trends) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/beverage-packaging-innovations-rising-trends landed on page that is not a product page. 2025-11-08 12:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-5-x-2-5-x-43-25-100-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/innovations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/happy-holidays-from-rocket-industrial returned 404 status code. 2025-11-08 12:58:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/sustainability already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/guide-to-a-cleaner-healthier-facility) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/guide-to-a-cleaner-healthier-facility landed on page that is not a product page. 2025-11-08 12:58:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/design already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-paper/35lb-vci-poly-coated-kraft-paper-roll-36-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/postal-approved-poly-strapping-kit.html returned 404 status code. 2025-11-08 12:58:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-paper/12-x-12-vci-kraft-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/60lb-vci-kraft-paper-roll-48-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/packaging-tips-for-new-businesses-and-startups returned 404 status code. 2025-11-08 12:58:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/advantages-inkjet-printers-have-over-labelers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-paper/35lb-vci-kraft-paper-roll-48-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/category/innovations) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/category/innovations landed on page that is not a product page. 2025-11-08 12:58:18 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 12:58:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-paper/60lb-vci-kraft-paper-roll-36-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/vci-packaging/vci-paper/35lb-vci-kraft-paper-roll-36-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/sustainability already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-12-160-edge-protector.html returned 404 status code. 2025-11-08 12:58:18 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/2-5-x-2-5-x-43-25-100-edge-protector.html>: HTTP status code is not handled or not allowed 2025-11-08 12:58:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/category/design) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/category/design landed on page that is not a product page. 2025-11-08 12:58:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/advantages-inkjet-printers-have-over-labelers) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/advantages-inkjet-printers-have-over-labelers landed on page that is not a product page. 2025-11-08 12:58:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-18-225-edge-protector.html returned 404 status code. 2025-11-08 12:58:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/nwd-betterwrapper.html returned 404 status code. 2025-11-08 12:58:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/jumbo-postal-approved-strapping-kit.html returned 404 status code. 2025-11-08 12:58:20 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/category/sustainability) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/category/sustainability landed on page that is not a product page. 2025-11-08 12:58:21 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-24-100-edge-protector.html returned 404 status code. 2025-11-08 12:58:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-mini-hand-stretch-film-120-ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-palletpal-360-spring-pallet-positioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-interpack-infeed-pack-exit-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fold-2-ply-paper-30-x-1155.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/pallet-stretch-wrap-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/superflex-45-gauge.html returned 404 status code. 2025-11-08 12:58:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-x-48-35-edge-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-inch-x-75-gauge-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ldx-rtb-4-series-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/evolve-63-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/pallet-stretch-wrap-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:26 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/resealable-antistatic-bag.html returned 404 status code. 2025-11-08 12:58:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-clear-resealable-bag-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:27 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/pink-amine-free-sealable-bag.html returned 404 status code. 2025-11-08 12:58:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ldx-rtb-4-series-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-4-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fox-fps300pa-portable-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000ebt-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-4-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-5-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-8-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-7-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-8-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-4-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-5-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-48-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:40 [py.warnings] (PID: 112) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2025-11-08 12:58:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:41 [scrapy.extensions.feedexport] (PID: 112) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:2 2025-11-08 12:58:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-42-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-42-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-40-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-9-x-27-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-14-x-36case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-42-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ecocraft-to-go-4-pound-deli-bag-vents.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-9-x-32case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-30-peach-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-heavy-duty-pivot-and-lock-casters.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-in-x-300-ft-barrier-tapes-caution.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blue-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-hot-melt-glue-stick-medium-set-5-8-inch-10-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-belt-lagging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dekka-2-inch-tape-head-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/industrial-plastic-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-adjustable-bag-opener.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-2-x-1000-yards-1-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-pre-stretch-roller-fg-03-03.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handle-it-model-800-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-3-x-18case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-gear-fg-03-12.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-rear-arm-spring-stud-cac50-022-3.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-order-picking-platform-cart-42x72.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:48 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/intertape-8100-heavy-duty-clear-machine-length-packaging-tape-2-x-1500-yards-2-2-mil.html returned 404 status code. 2025-11-08 12:58:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-26-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:49 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3m-basic-tape-dispenser.html returned 404 status code. 2025-11-08 12:58:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/custom-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sweed-model-517-xhd-scrap-chopper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/alpha-hsm-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sharp-max-20-roll-bagging-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sweed-cl200-cut-to-length-strapping-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/polyester-vs-polypropylene-strapping-whats-the-difference already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/custom-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/services/packaging-test-lab.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/secure-shipments-for-less-with-the-ri-200-strapping-tool already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-60-brown-duct-tape-10.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-duct-tape-36.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/rocket already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-100-black-poly-sheeting-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-duct-tape-10.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/robots-automation-take-jobs returned 404 status code. 2025-11-08 12:58:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-bob-schymanski returned 404 status code. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/white-duct-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-60-yds-duct-tape-20.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-2979-contractor-grade-duct-tape-2-x-60-yds-7-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/stretch-wrapping/stretch-wrappers/semi-automatic-stretch-wrappers.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/custom-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/custom-packaging.html landed on page that is not a product page. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000ebm-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/polyester-vs-polypropylene-strapping-whats-the-difference) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/polyester-vs-polypropylene-strapping-whats-the-difference landed on page that is not a product page. 2025-11-08 12:58:51 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/eco-viewing-our-favorite-nature-documentaries returned 404 status code. 2025-11-08 12:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000b-hp-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handle-it-model-1100-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-low-profile-2100-srt-semi-auto-pallet-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/services/packaging-test-lab.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-2000b-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/secure-shipments-for-less-with-the-ri-200-strapping-tool) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/secure-shipments-for-less-with-the-ri-200-strapping-tool landed on page that is not a product page. 2025-11-08 12:58:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/category/rocket) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/category/rocket landed on page that is not a product page. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-ecoplat-stretch-wrapper-86-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/black-duct-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-x-040-x-4000-green-polyester-smooth-tool-grade-strapping-1600-lb-16-x-6-core.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-x-030-x-4600-green-polyester-smooth-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sweed-model-517-xhd-scrap-chopper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:52 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ideas/ebooks/brewers-packaging-guide.html returned 404 status code. 2025-11-08 12:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/services/packaging-test-lab.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:54 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/services/packaging-test-lab.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/services/packaging-test-lab.html landed on page that is not a product page. 2025-11-08 12:58:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-43-5-vboard-edge-protectors.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/stretch-wrapping/stretch-wrappers/semi-automatic-stretch-wrappers.html?p=3 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martego-aluminum-smart-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handle-it-model-850ps-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-masterwrap-hd-xl-rotary-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-secunorm-profi-25-semi-auto-retractable-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/battery-for-orgapack-250-strapping-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-rotoplat-708-semi-automatic-stretch-wrapper-86-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/extended-tower-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nestaflex-226-flexible-gravity-conveyor.html#579=9472 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-technoplat-708-cs-semi-automatic-stretch-wrapper-86-height.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/amine-free-resealable-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/the-pilot-semi-auto-stretch-wrapper-low-profile-turntable.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fox-manual-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handle-it-model-1200-ul-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:57 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/20-inch-amtopp-70-gauge-high-performance-stretch-wrap-6500-foot-rolls.html returned 404 status code. 2025-11-08 12:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-1000a-manual-pre-stretch-pallet-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-rotoplat-708-semi-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-intertape-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sabert-black-medium-24-oz-square-bowl.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x6000-intertape-75-gauge-genesys-superflex-machine-stretch-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-cobra-g-rotary-arm-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cousins-relay-120-vac-coil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-2-inch-reclosable-bags-2-mil4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-8-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-14clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:58:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-5-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-3-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-4-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-4-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-7-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-48-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 4282 pages (at 371 pages/min), scraped 2096 items (at 162 items/min) 2025-11-08 12:59:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-24-x-60-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-36-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-15-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-clear-lids-epclid3-case-of-2500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-60-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-9-x-24-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-3-x-15case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-ld7-belt-set.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-in-x-60-yd-filament-tape-rg15.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-2-x-1500-yards-1-6-mil.html returned 404 status code. 2025-11-08 12:59:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-5490-ptfe-film-tape-1-x-36.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cac50-loveshaw-tape-head-cartridge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/airmove-void-air-pillows.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-9-x-24-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-drive-belt-n136-ac.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-drive-belt-ldx-0048b-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-5-x-4-25-kraft-gummed-envelopes.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-transmission-shaft-fj-1a-200.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-conveyor-bearing-6005.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-9-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-10-x-36-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-spkt-ldxrtb-60.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/clysar-hpg-shrink-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-industrial-warehouse-trailer-36x72.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/madison-tm-21-orbital-tape-bander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-zls-low-profile-lift-table-6000-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:24 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/4-6-white-perforated-transfer-labels.html returned 404 status code. 2025-11-08 12:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100-knife-blade-assembly.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5300-tamp-blow-label-printer-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30lb-kraft-paper-roll-48x1200.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/nutting-industrial-warehouse-trailer-36x72.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg16-machine-length-tape-2-x-1000.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-infeed-roller-psc301211-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/air-bubble-rolls-1-2-24-125.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/open-positions.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40lb-kraft-paper-roll-36x900.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-abal-infeed-outfeed-pack-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/our-promise.html returned 404 status code. 2025-11-08 12:59:26 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/open-positions.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/open-positions.html landed on page that is not a product page. 2025-11-08 12:59:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/must-have-tools-for-your-warehouse already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/5-16-x-24-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/influence-packaging-brand-image returned 404 status code. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/3-16-x-24-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heavy-duty-tension-device.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-5000-zenith-mid-machine-film-57-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/stretch-wrapping/stretch-wrap/machine-stretch-wrap.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/genesys-20-inch-machine-stretch-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ri-200-30-day-guarantee.html returned 404 status code. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ipg-stretchflex-sf1-75-gauge-stretch-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-march already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-genesis-thunder-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-power-flex-tl-stretch-hooder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:27 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/must-have-tools-for-your-warehouse) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/must-have-tools-for-your-warehouse landed on page that is not a product page. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-industrial already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/standard-loading-ramp.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/food-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/jumbo-general-purpose-strapping-kit.html returned 404 status code. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-power-flex-t1-stretch-hooder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-multi-flexl-stretch-hooder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-3500-zenith-mid-machine-film-150-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zenith-high-performace-machine-stretch-wrap-70-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-march) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-march landed on page that is not a product page. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zenith-high-performance-machine-stretch-film-75-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zenith-ultra-performance-machine-stretch-film-41-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-5000-zenith-mid-machine-film-115-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-industrial) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-industrial landed on page that is not a product page. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-5000-feet-zenith-high-performance-machine-stretch-wrap-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zenith-high-performance-machine-stretch-wrap-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zenith-ultra-performance-machine-stretch-film-55-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-x-75-gauge-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-6000-zenith-mid-machine-film-90-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/food-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/food-packaging.html landed on page that is not a product page. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-5500-zenith-mid-machine-film-100-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/signode-multi-flex1-stretch-hooder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/automatic-conveyor-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/high-profile-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-helix-1-evo-automatic-rotary-arm-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/robopac-rotoplat-3000-hd-automatic-stretch-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zenith-ultra-performance-machine-stretch-film-50-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-6000-zenith-machine-stretch-film-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-6500-zenith-machine-stretch-film-70-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-5000-zenith-mid-machine-film-80-gauge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-1325-battery-strapping-tensioner-1-1-1-2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-1345-battery-powered-cord-strap-tensioner-with-cutter.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/hd-jumbo-poly-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/hd-jumbo-cord-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/round-load-tensioner.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/industrial-ratchet-and-cut-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heavy-duty-steel-sealless-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/std-duty-steel-sealless-tool.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/1-2-x-24-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/5-16-x-12-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/1-2-x-12-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/air-bubble-rolls-3-16-12-300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/air-bubble-rolls-3-16-24-300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/air-bubble-rolls-1-2-24-125.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/packaging-equipment-financing-payment-alternatives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/bubble-cushioning/bubble-rolls/5-16-x-12-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/packaging-equipment-financing-payment-alternatives) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/packaging-equipment-financing-payment-alternatives landed on page that is not a product page. 2025-11-08 12:59:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ipg-stretchflex-sf1-80-gauge-stretch-wrap.html returned 404 status code. 2025-11-08 12:59:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-70-gauge.html returned 404 status code. 2025-11-08 12:59:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dart-staylock-clear-pet-plastic-square-hinged-lid-deli-containers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:46 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/17-inch-amtopp-pallet-lock-38-gauge-hand-stretch-film-1476-foot-rolls.html returned 404 status code. 2025-11-08 12:59:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/yellow-safety-grip-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:46 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/40lb-vci-kraft-paper-roll-36-x-200yd.html returned 404 status code. 2025-11-08 12:59:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/40lb-vci-kraft-paper-roll-48-x-200yd.html returned 404 status code. 2025-11-08 12:59:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-6-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-5-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-5-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-36-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-30-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-60-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:58 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-40-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-16-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1000-yard-clear-industrial-packing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-10-x-32-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 12:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-44-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-am591-hot-melt-glue-sticks-medium-fast-set-5-8-inch-10-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/pti-2-inch-x-1000-yard-clear-machine-length-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/secumax-replacement-cutting-heads.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cac60-tape-head-loveshaw.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-8-x-24-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/52-x-60-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-5-x-14-5-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-18-x-36-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-5-x-16-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/15-5-x-9-875-curby-mailer.html returned 404 status code. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/45-x-37-grip-sheet-anti-slip-pallet-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/rocket-industrial-180-high-temp-hot-melt-glue-gun-1-2-inch.html returned 404 status code. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rollbag-r785-automatic-bagger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-100-black-poly-sheeting-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-manual-portable-dandy-lift-table-1760-lbs.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/beverages.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/case-studies.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wexxar-wf30-fully-automatic-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-css-automatic-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/finance-application.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-5-x-16-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-cylinder-repair-kit-ld12b-2048r.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/beverages.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/beverages.html landed on page that is not a product page. 2025-11-08 13:00:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/case-studies.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/case-studies.html landed on page that is not a product page. 2025-11-08 13:00:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/how-to-load-a-tape-dispenser.html returned 404 status code. 2025-11-08 13:00:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/finance-application.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/finance-application.html landed on page that is not a product page. 2025-11-08 13:00:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/packaging-wrap-up-january-2021 returned 404 status code. 2025-11-08 13:00:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/matting-programs-are-a-winning-strategy-for-industrial-athletes returned 404 status code. 2025-11-08 13:00:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/operation-packaging-care-2017 returned 404 status code. 2025-11-08 13:00:06 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 13:00:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-battery-charger-b400-b800.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:06 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/stretch-wrapper-advantage returned 404 status code. 2025-11-08 13:00:06 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/10-5-x-16-kraft-self-seal-mailer.html>: HTTP status code is not handled or not allowed 2025-11-08 13:00:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ep-800-pallet-wrap-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:06 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/this-month-in-packaging-february-2021 returned 404 status code. 2025-11-08 13:00:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-48-160-edge-protector.html returned 404 status code. 2025-11-08 13:00:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-oz-red-plaid-paper-food-tray.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-30-225-edge-protector.html returned 404 status code. 2025-11-08 13:00:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-lb-kraft-paper-food-trays.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-smoke-fan-110v.html returned 404 status code. 2025-11-08 13:00:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/charger-for-orgapack-50-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 4601 pages (at 319 pages/min), scraped 2238 items (at 142 items/min) 2025-11-08 13:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stationary-cord-strapping-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-airspace-bubble-on-demand-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-24-x-48-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/green-safety-grip-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-15-flairpak-500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-8-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-18-clear-wicketed-bread-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-15-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-9-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-28-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-4-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-6-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-6-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-3-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-18-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-20-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-8-x-24-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/52-x-60-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-2-x-10case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-26-x-60case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-15-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-24-x-48-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-in-x-60-yd-flatback-paper-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/quad-701-fast-set-hot-melt-glue-stick.html returned 404 status code. 2025-11-08 13:00:25 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/6-x-12-black-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 13:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-18-x-48-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-in-x-60-yd-masking-tape-intertape-pg505.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-in-x-60-yd-intertape-534-flatback-paper-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-7151qt-medium-duty-cold-temp-clear-machine-length-packaging-tape-2-x-1500-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/full-face-5-5-inch-orange-packing-list.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-impulse-bag-sealer-with-cutter.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b600-strapping-tool-feed-wheel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-pack-exit-table-36-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-orangemask-premium-grade-masking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-strapping-tool-battery-charger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-tape-head-drum-fj-02-05-03.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-x-26-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-return-spring-fj-h-00.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/madison-tm-17-orbital-tape-bander.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-tape-head-drum-fj-02-05-03.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-anti-skid-washer-fj-05-01.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-3510-handheld-twist-tie-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-motor-starter-1-pole-psc636-ab-2.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-in-x-60-yd-filament-tape-rg300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vci-yellow-gusseted-poly-bags-34-x-31-x-69-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-24-x-48-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/airmove-cushion-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-palletpal-turntable-disc.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/apply-business-account.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/zerust-vc1-1-vci-vapor-capsules.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/building-products-materials-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cannabis-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:31 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ideas/ebooks/cold-chain-packaging.html returned 404 status code. 2025-11-08 13:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/packaging-test-lab.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-8-in-x-60-yd-flatback-paper-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/minneapolis.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:32 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/apply-business-account.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/apply-business-account.html landed on page that is not a product page. 2025-11-08 13:00:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sponsorship-donations.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:32 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/building-products-materials-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/building-products-materials-packaging.html landed on page that is not a product page. 2025-11-08 13:00:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/why-choose-a-career-in-packaging already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:32 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/cannabis-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/cannabis-packaging.html landed on page that is not a product page. 2025-11-08 13:00:32 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/packaging-test-lab.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/packaging-test-lab.html landed on page that is not a product page. 2025-11-08 13:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/letter-from-president already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-2-inch-case-sealer-tape-head-1.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/5-tips-for-safely-shipping-automotive-parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/minneapolis.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:33 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/sponsorship-donations.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/sponsorship-donations.html landed on page that is not a product page. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/in-memory-of-tommy-wanserski already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/choosing-the-right-palletizer-for-your-production-line already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-cylinder-repair-kit-n211a-nor-r.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/why-choose-a-career-in-packaging) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/why-choose-a-career-in-packaging landed on page that is not a product page. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sweed-510-xhd-scrap-chopper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-april already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/letter-from-president) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/letter-from-president landed on page that is not a product page. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/shrink-sealing-heat-gun-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/case-sealers-erectors/case-sealers/random-case-sealers.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/5-tips-for-safely-shipping-automotive-parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-csf-automatic-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:34 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/in-memory-of-tommy-wanserski) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/in-memory-of-tommy-wanserski landed on page that is not a product page. 2025-11-08 13:00:34 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/contact-us/minneapolis.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/contact-us/minneapolis.html landed on page that is not a product page. 2025-11-08 13:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-ct4e-automatic-random-four-edges-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t400rl-random-large-box-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-rq22-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:35 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-april) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-april landed on page that is not a product page. 2025-11-08 13:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/stretch-wrapping/stretch-wrap-dispensers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-gun-shrink-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b400-replacement-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3m-200a-case-sealer-rfb.html returned 404 status code. 2025-11-08 13:00:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/employee-spotlight-lauri returned 404 status code. 2025-11-08 13:00:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/5-tips-for-safely-shipping-automotive-parts) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/5-tips-for-safely-shipping-automotive-parts landed on page that is not a product page. 2025-11-08 13:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/choosing-the-right-palletizer-for-your-production-line already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/silicone-release-agent.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-7000r3-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-800rks-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bottom-chuck-modified.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-8000r-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-matic-800r3-random-case-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-allfit-blunt-tip-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/choosing-the-right-palletizer-for-your-production-line) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/choosing-the-right-palletizer-for-your-production-line landed on page that is not a product page. 2025-11-08 13:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/small-ripper-9.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-835-hand-stretch-wrap-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-790-wrapstik.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/superflex-39-gauge.html returned 404 status code. 2025-11-08 13:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/35lb-vci-kraft-paper-roll-36-x-200yd.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/poly-strap-disenser-strap-troller.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/catalog/product/view/id/5289 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-36-225-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/anti-static-esd/static-shielding-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-pre-opened-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/handi-foil-12-inch-x-5-inch-recyclable-aluminum-oblong-danish-pan.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/catalog/product/view/id/5289 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-15-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-pre-opened-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:50 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 13:00:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:50 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/catalog/product/view/id/5289>: HTTP status code is not handled or not allowed 2025-11-08 13:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-clear-poly-tubing-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-6-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-6-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:00:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-10-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-54-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-15-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-42-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/44-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-2-x-12-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-4-x-15case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-8-x-20-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-6-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-60-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-5-x-13-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-swivel-and-lock-casters.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/34-x-40-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-friction-reduction-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 4926 pages (at 325 pages/min), scraped 2381 items (at 143 items/min) 2025-11-08 13:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rocket-industrial-hot-melt-glue-stick-medium-set-1-2-inch-10-inch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:09 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/8-x-12-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-18-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-pivot-shaft.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/gray-colored-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-kraft-self-seal-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-kraft-self-seal-stayflatsr-mailer-6-x-6.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-4000-lbs-lift-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/printer-to-taper-mounting-plate-ld16a-ldu.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-rpk-16a-cac60.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-shaft-fj-00-02.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/berran-replacement-tape-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-40-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-36-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-red-vinyl-tape-2-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-pin-release-bar-assembly-fg-08-19a.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-kraft-self-seal-stayflatsr-mailer-6-x-6.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-interpack-standard-tape-head-1.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-earth-epphl-93-carryout-food-containers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/polyken-510-red-gaffers-tape-56-x-55.html returned 404 status code. 2025-11-08 13:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100sm-rear-idler-wheel-roller-fj-40-03-sm.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/account-business-application.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/best-pack-elvs-case-erector-bottom-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/efficient-crating.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rollbag-r3200-high-speed-automatic-bagger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/survivor-cut-resistant-gloves-13.html returned 404 status code. 2025-11-08 13:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/best-way-protect-product-design-for-distribution already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/plastic-trash-turned-to-art returned 404 status code. 2025-11-08 13:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/account-business-application.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/secret-to-lower-stretch-film-expenses-package-with-wes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/introducing-project100k returned 404 status code. 2025-11-08 13:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-september already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-inch-interpack-standard-tape-head-1.html returned 404 status code. 2025-11-08 13:01:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/efficient-crating.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/efficient-crating.html landed on page that is not a product page. 2025-11-08 13:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-lm100-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-industrial-is-great-place-to-work-certified-for-2023 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/best-way-protect-product-design-for-distribution) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/best-way-protect-product-design-for-distribution landed on page that is not a product page. 2025-11-08 13:01:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/account-business-application.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/account-business-application.html landed on page that is not a product page. 2025-11-08 13:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3inch-by-5inch-anti-static-reclosable-poly-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13inch-by-18inch-anti-static-reclosable-poly-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/a-complete-guide-to-different-types-of-tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/secret-to-lower-stretch-film-expenses-package-with-wes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-september) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-september landed on page that is not a product page. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/foam-packaging/packing-peanuts/eco-friendly-packing-peanuts.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6x-10-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-7-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-black-backed-vacuum-pouch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12inch-by-12inch-anti-static-reclosable-poly-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cord-strapping-cart-dispenser.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9inch-by-12inch-anti-static-reclosable-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-industrial-is-great-place-to-work-certified-for-2023) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-industrial-is-great-place-to-work-certified-for-2023 landed on page that is not a product page. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b1200-battery.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8inch-by-10inch-anti-static-reclosable-poly-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/a-complete-guide-to-different-types-of-tapes) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/a-complete-guide-to-different-types-of-tapes landed on page that is not a product page. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6inch-by-8inch-anti-static-reclosable-poly-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-13-safe-handling-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/secret-to-lower-stretch-film-expenses-package-with-wes) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/secret-to-lower-stretch-film-expenses-package-with-wes landed on page that is not a product page. 2025-11-08 13:01:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4inch-by-6inch-anti-static-reclosable-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/trends-tips-for-cannabis-packaging-design already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-40-225-edge-protector.html returned 404 status code. 2025-11-08 13:01:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9inch-by-12inch-anti-static-reclosable-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/ray-and-marie-goldbach-business-principles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:22 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/trends-tips-for-cannabis-packaging-design) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/trends-tips-for-cannabis-packaging-design landed on page that is not a product page. 2025-11-08 13:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/klever-koncept-kcj-2-yellow-safety-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:22 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/ray-and-marie-goldbach-business-principles) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/ray-and-marie-goldbach-business-principles landed on page that is not a product page. 2025-11-08 13:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sp4-1-black-ink-cartridge-for-the-anser-u2-smartone-1-inch-thermal-inkjet-printer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/merak-trigger-smart-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/knife-replacement-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stamp-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bottle-labeling-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/switch-with-key.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dw-fine-pack-large-rotisserie-inter-lock-containers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/print-and-apply-labeling-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-trusted-lock-freezer-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-12-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dyne-a-pak-black-foam-meat-trays-3pp.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-30-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-48-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-6-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-20-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-6-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-18-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-36-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-42-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-7-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-3-x-12case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-30-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-black-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ecocraft-hot-to-go-window-deli-bag-w-vents-xl.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:46 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/6-x-8-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 13:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/22-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-30-green-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-x-18-yd-ptfe-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-inch-x-1500-3m-371-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bar-print-5-5-inch-orange-packing-list.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/microjet-to-sealer-mount-box-ld7.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-in-x-60-yd-masking-tape-intertape-pg505.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:53 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-tension-spring.html returned 404 status code. 2025-11-08 13:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-black-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8oz-hot-styrofoam-cups.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-3570-semi-automatic-twist-tie-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-3568-automatic-twist-tie-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:55 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/safety-aisle-tape-applicator.html returned 404 status code. 2025-11-08 13:01:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-x-60-yd-masking-tape-intertape-pg505.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/take-a-label-2100er-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100sm-knife-blade-assembly.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/pilot-semi-auto-stretch-wrapper-low-profile-turntable-demo.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/no-mess-tubing-closer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b1200-strapping-tool-feed-wheel.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-3-inch-top-tension-roller-78-8054-8797-8.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/51-49-85-3-mil-box-bin-liners.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:01:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/careers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eastey-shrink-combo.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/polyken-105c-double-coated-cloth-tape-60-x-25.html returned 404 status code. 2025-11-08 13:02:00 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ideas/ebooks/packaging-automation-ebook.html returned 404 status code. 2025-11-08 13:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-1000-white-unwaxed-butcher-paper-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/how-to-properly-remove-disposable-gloves returned 404 status code. 2025-11-08 13:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/25ft-soft-wire-19ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/50ft-half-hard-wire-19ga.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:02 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/careers.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/careers.html landed on page that is not a product page. 2025-11-08 13:02:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/sustainable-packaging-aquatic-life.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wrap-cutoff-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-x-75-gauge-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:03 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-60-160-edge-protector.html returned 404 status code. 2025-11-08 13:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/gaylord-stand-for-the-sweed-300-450-strap-choppers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/hopper-stand-for-the-sweed-300-450-strap-choppers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:03 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/sustainable-packaging-aquatic-life.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/sustainable-packaging-aquatic-life.html landed on page that is not a product page. 2025-11-08 13:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sweed-ez-dump-hopper-1-2-yard.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-18-kraft-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4300-tamp-blow-label-printer-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-trusted-ultra-guard-locker-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ripper-16.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/samuel-drive-wheel.html returned 404 status code. 2025-11-08 13:02:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-90-gauge.html returned 404 status code. 2025-11-08 13:02:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/static-protected-resealable-bag.html returned 404 status code. 2025-11-08 13:02:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-12-pre-opened-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-22-flairpak-400.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:06 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-60-gauge.html returned 404 status code. 2025-11-08 13:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/20-inch-nexus-machine-film-80-gauge-stretch-wrap-6000-foot-rolls.html returned 404 status code. 2025-11-08 13:02:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 5232 pages (at 306 pages/min), scraped 2503 items (at 122 items/min) 2025-11-08 13:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-inch-powered-roller-conveyor-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-18-gusseted-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-8-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-48-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-18-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-3-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-28-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-48-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-8-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-16-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-14-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-4-x-20case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-x-1000-yard-clear-packaging-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/filling-table-ld15-ld19pt-tapers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-mbd-series-case-sealer-belts-2x2050mm.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-14-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/15-x-9-x-32-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/texwrap-st-2215r-spartan-automatic-in-line-l-bar-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-orientation-plate-fg-03a-20.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eastey-em1636t-performance-series-manual-hot-wire-l-bar-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-r-automatic-label-applicator.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-drive-belt-27100240.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-spare-parts-kit-rpk-16sb-st.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-cylinder-n401-454.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-idler-bearing-60202.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/clysar-evo-recyclable-shrink-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/best-pack-mbf-packing-station.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-in-blades-for-interpack-case-sealers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polychem-b1200-strapping-tool-feed-wheel-1.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-2-inch-tape-head-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-cf-25-drive-belt-set-ld3sb2-2004u-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-2-inch-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-100-clear-poly-sheeting-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/air-bubble-rolls-3-16-24-300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-bg18-machine-length-tape-3-x-1000.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-drive-belt-1520mmx50x5mm.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/mastermover-sm100-electric-tugger.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3m-brush-assembly-78-8060-7936-0.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-75-x-47-mailing-tubes.html returned 404 status code. 2025-11-08 13:02:36 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/contact-us.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/contact-us.html landed on page that is not a product page. 2025-11-08 13:02:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/stretch-wrap-usage-and-buying-guide already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ideas/case-studies/shipping-focused-packaging.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/tape/carton-sealing-tape/machine-carton-sealing-tape.html?p=2 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-to-use-water-activated-tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-seal-hse-30a-compact-shrink-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-seal-hse-50a-one-step-shrink-wrapper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/corrugated-box-basics already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/newsletter.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/sustainability-terms-you-need-to-know already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-seal-hdx-250a-automatic-combo-shrink-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/ideas/case-studies/shipping-focused-packaging.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/ideas/case-studies/shipping-focused-packaging.html landed on page that is not a product page. 2025-11-08 13:02:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/cost-efficient-window-and-door-packaging-solutions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-to-use-water-activated-tape) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-to-use-water-activated-tape landed on page that is not a product page. 2025-11-08 13:02:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/stretch-wrap-usage-and-buying-guide) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/stretch-wrap-usage-and-buying-guide landed on page that is not a product page. 2025-11-08 13:02:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/corrugated-box-basics) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/corrugated-box-basics landed on page that is not a product page. 2025-11-08 13:02:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/newsletter.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/newsletter.html landed on page that is not a product page. 2025-11-08 13:02:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/sustainability-terms-you-need-to-know) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/sustainability-terms-you-need-to-know landed on page that is not a product page. 2025-11-08 13:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/foam-packaging/foam-pouches-bags/premier-protective-packaging-white-polyethylene-foam-pouches.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-40-120-edge-protector.html returned 404 status code. 2025-11-08 13:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-airspace-g1-air-pillow-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/cost-efficient-window-and-door-packaging-solutions) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/cost-efficient-window-and-door-packaging-solutions landed on page that is not a product page. 2025-11-08 13:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wrap-cycle-wireless-start-remote.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/15-inch-extended-core-stretch-wrap.html returned 404 status code. 2025-11-08 13:02:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-36-120-edge-protector.html returned 404 status code. 2025-11-08 13:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/die-cast-metal-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-2-x-12-bubble-dispenser-pack.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:45 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/18-inch-cast-hand-stretch-wrap.html returned 404 status code. 2025-11-08 13:02:45 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/3-x-3-x-18-120-edge-protector.html returned 404 status code. 2025-11-08 13:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-short-infeed-pack-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-3-x-1000-yards-1-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/sabert-7-5-x-7-5-clear-dome-lid.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/die-cast-metal-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/safety-cutter-knife-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-oz-clear-plastic-deli-container.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/polyair-airspace-air-pillow-film-8-x-5-inch-roll.html returned 404 status code. 2025-11-08 13:02:48 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/combi-ce-10-carton-erector-bottom-case-sealer.html returned 404 status code. 2025-11-08 13:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-15-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/die-cast-metal-utility-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-ergopack-250-hand-packing-station.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-4-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:48 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-18-160-edge-protector.html returned 404 status code. 2025-11-08 13:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-6-clear-resealable-bag-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-7-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-10-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:49 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 429 Unknown Status 2025-11-08 13:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:49 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <429 https://www.rocketindustrial.com/die-cast-metal-utility-knife.html>: HTTP status code is not handled or not allowed 2025-11-08 13:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-5-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-7-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-8-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-14-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-20-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/13-x-13-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-clear-poly-tubing-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-18-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-32-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-12-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-24-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-18-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:02:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-2-x-12case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-10-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-8-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:01 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/10-x-12-black-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 13:03:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-inch-x-36-yard-heat-sealing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/palletpod-compact-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-30-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-in-x-66-ft-electrical-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-3-flap-folder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/tape/cloth-tape/duct-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/4-x-9-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 13:03:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/support-kit-24-inch-industrial-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-mini-con-r-with-feed-table.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/tach-it-lr500-label-rewinder.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-in-x-60-yd-filament-tape-rg300.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/good-natured-80-gauge-machine-stretch-wrap.html returned 404 status code. 2025-11-08 13:03:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100sm-front-idler-wheel-roller-fj-e-41-01-sm.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/taconic-6445-05-high-modulus-ptfe-film-tape-24-x-36.html returned 404 status code. 2025-11-08 13:03:06 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/packlytics-semi-automatic-smart-stretch-wrapper.html returned 404 status code. 2025-11-08 13:03:07 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-8-x-035-x-4200-green-polyester-smooth-strapping.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/air-bubble-roll-3-16-48-750.html returned 404 status code. 2025-11-08 13:03:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-12-x-36case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/industries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 5515 pages (at 283 pages/min), scraped 2612 items (at 109 items/min) 2025-11-08 13:03:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/craft-beer-packaging-trends already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/enable-cookies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:10 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/taconic-6115-03-skived-ptfe-tape-36-x-36.html returned 404 status code. 2025-11-08 13:03:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/request-catalog.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/how-did-the-green-bay-packers-get-their-name-timeline already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/tape-vs-glue-whats-best-for-you already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/1-inch-x-60-yds-high-temp-masking-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/service-repair.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:10 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/ipg-american-made-tapes-films-packaging returned 404 status code. 2025-11-08 13:03:10 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/industries) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/industries landed on page that is not a product page. 2025-11-08 13:03:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/craft-beer-packaging-trends) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/craft-beer-packaging-trends landed on page that is not a product page. 2025-11-08 13:03:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/enable-cookies) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/enable-cookies landed on page that is not a product page. 2025-11-08 13:03:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/uses-and-benefits-of-chamber-vacuum-sealer-bags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/request-catalog.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/request-catalog.html landed on page that is not a product page. 2025-11-08 13:03:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/how-did-the-green-bay-packers-get-their-name-timeline) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/how-did-the-green-bay-packers-get-their-name-timeline landed on page that is not a product page. 2025-11-08 13:03:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/airmove-bubble-cushioning-film.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/service-repair.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/service-repair.html landed on page that is not a product page. 2025-11-08 13:03:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/products/protective-packaging/paper-void-fill/paper-rolls/40lb-kraft-paper-roll-24x900.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-600-60-lb-kraft-paper-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/category/equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/uses-and-benefits-of-chamber-vacuum-sealer-bags) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/uses-and-benefits-of-chamber-vacuum-sealer-bags landed on page that is not a product page. 2025-11-08 13:03:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-1200-30-lb-kraft-paper-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/tape-vs-glue-whats-best-for-you) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/tape-vs-glue-whats-best-for-you landed on page that is not a product page. 2025-11-08 13:03:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/spare-parts-kit-for-automatic-wrappers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/creating-brand-engagement-with-custom-packaging already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/southworth-palletpal-pallet-inverter.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/fromm-replacement-battery-p328-p329.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/category/equipment) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/category/equipment landed on page that is not a product page. 2025-11-08 13:03:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-secupro-625-fully-automatic-retractable-blade-safety-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/polyair-x-fold-2-ply-paper-30-x-990.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/rocket-women-in-packaging already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/creating-brand-engagement-with-custom-packaging) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/creating-brand-engagement-with-custom-packaging landed on page that is not a product page. 2025-11-08 13:03:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/everything-you-need-to-know-about-scrap-choppers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-x-2-x-6-225-strapping-protectors.html returned 404 status code. 2025-11-08 13:03:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/20-inch-cold-force-machine-film-71-gauge-stretch-wrap-6500-foot-rolls.html returned 404 status code. 2025-11-08 13:03:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/4-x-4-x-2-160-strapping-protectors.html returned 404 status code. 2025-11-08 13:03:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/rocket-women-in-packaging) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/rocket-women-in-packaging landed on page that is not a product page. 2025-11-08 13:03:18 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/everything-you-need-to-know-about-scrap-choppers) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/everything-you-need-to-know-about-scrap-choppers landed on page that is not a product page. 2025-11-08 13:03:18 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/detroit-forming-ops-plastic-clear-locking-hinged-bakery-container.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-18-x-36-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:19 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/1-2-x-12-static-control-bubble-packaging.html returned 404 status code. 2025-11-08 13:03:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-12-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-inch-clear-poly-tubing-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:20 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10x8x24-gusseted-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:21 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-24-case-packed-flat-bag-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-24-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:22 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-36-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-12-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-30-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-8-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:23 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-5-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:24 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-12-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:25 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-26-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:26 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:27 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-6-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:28 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-9-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-26-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-48-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:29 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-18-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:30 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/10-x-36-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:31 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-4-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-8-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:32 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-42-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-18-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-24-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-4-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/44-x-48-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-56-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/26-x-36-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/16-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-42-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-22-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-10-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/40-x-60-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-16-x-42case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5oz-water-cup.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:36 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-8-orange-unitizing-tape.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-30-green-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-tape-cartridge-cac16a1.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:37 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-siderail-cap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:38 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-clear-portion-cups-epc325-case-of-2500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:38 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-3-x-1500-yards-1-6-mil.html returned 404 status code. 2025-11-08 13:03:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/32-x-40-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-4-in-x-60-yd-filament-tape-rg316.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-25-x-20-kraft-self-seal-bubble-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-extended-film-roller-fg-08-08.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:39 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/intertape-8100-heavy-duty-clear-machine-length-packaging-tape-3-x-1500-yards-2-2-mil.html returned 404 status code. 2025-11-08 13:03:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/foot-pedal-for-airmove2-air-pillow-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:39 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/20-x-18-x-36-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-secumax-320-concealed-blade-safety-knife.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-knob-fjg-1a-197sw.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:40 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/service-kit-4-inch-plastic-heat-sealer.html returned 404 status code. 2025-11-08 13:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-25-x-20-kraft-self-seal-bubble-mailer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/columbia-machine-fl2000-floor-level-palletizer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/ri-200-strapping-tool-feed-wheel-1.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/encore-ep-3550-tool-balancer-cart-standard.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/rocket-industrial-low-temp-hot-melt-glue-gun.html returned 404 status code. 2025-11-08 13:03:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ipg-591-flatback-paper-tape-23-x-36-12-mil.html returned 404 status code. 2025-11-08 13:03:41 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/atp-dc-4420lb-double-coated-pvc-tape-39-x-60.html returned 404 status code. 2025-11-08 13:03:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/madison.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:41 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/submit-to-the-monthly-packaging-wrap-up.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:41 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 13:03:41 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/14-25-x-20-kraft-self-seal-bubble-mailer.html>: HTTP status code is not handled or not allowed 2025-11-08 13:03:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/4-manufacturers-going-zero-waste-to-landfill returned 404 status code. 2025-11-08 13:03:42 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/strapping-banding-tool-repair.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:42 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/super-bowl-going-plastic-free returned 404 status code. 2025-11-08 13:03:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ipg-bt100-blast-tape-log.html returned 404 status code. 2025-11-08 13:03:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/submit-to-the-monthly-packaging-wrap-up.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/submit-to-the-monthly-packaging-wrap-up.html landed on page that is not a product page. 2025-11-08 13:03:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/madison.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/sustainable-packaging-stats-you-should-know already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/stretch-dancer-spring-switch.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-inch-intertape-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/eagle-q31-19-battery-powered-strapping-tool.html returned 404 status code. 2025-11-08 13:03:43 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/strapping-banding-tool-repair.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/strapping-banding-tool-repair.html landed on page that is not a product page. 2025-11-08 13:03:43 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/martor-styropor-blade.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dart-solo-16-oz-translucent-plastic-party-cups.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:44 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-2ez-high-speed-case-erector-with-bottom-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/contact-us/madison.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/contact-us/madison.html landed on page that is not a product page. 2025-11-08 13:03:44 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/sustainable-packaging-stats-you-should-know) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/sustainable-packaging-stats-you-should-know landed on page that is not a product page. 2025-11-08 13:03:45 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-40-160-edge-protector.html returned 404 status code. 2025-11-08 13:03:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/dart-solo-16-oz-translucent-plastic-party-cups.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:45 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-t100sm-2-inch-tape-specialty-case-sealer-heads.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:46 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/stretchflex-stretch-wrap-80-gauge.html returned 404 status code. 2025-11-08 13:03:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-alphapack-250-compact-case-packing-system.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-80-gauge-hand-stretch-film-1000-foot-rolls.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:46 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-inch-clear-poly-tubing-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:47 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/nelson-wrap-dispenser.html returned 404 status code. 2025-11-08 13:03:47 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-8-x-30-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-inch-trusted-ultra-guard-locker-paper.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-18-flat-bags-on-a-roll.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-22-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:48 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-20-inch-reclosable-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-inch-shrink-wrap.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-20-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/8-x-4-x-22-case-packed-gusseted-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-16-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:50 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-4-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-18-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:51 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-16-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:52 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/18-x-24-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-48-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/48-x-54-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:53 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-x-30-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:54 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:55 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/7-x-9-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/14-x-14-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/24-x-30-case-packed-flat-bags-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:56 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-14-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-x-12-case-packed-flat-bags-4-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/36-x-36-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:57 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-3-x-15-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-x-3-x-15-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-42-clear-layflat-poly-bags-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-42-case-packed-flat-bags-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/44-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:03:59 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-16-shrink-bag.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/6-x-30-black-steak-paper-sheets.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/3-inch-bestpack-tape-head.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/empress-clear-lids-epclid2-case-of-2500.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/44-x-60-case-packed-flat-bags-2-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:00 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/11-x-16-case-packed-flat-bags-1-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:01 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <403 https://www.rocketindustrial.com/1850x2-black-belt.html>: HTTP status code is not handled or not allowed 2025-11-08 13:04:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/9-x-12-clear-layflat-poly-bags-1-25-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:01 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/front-rubber-roller-fj-e-41-01-for-eagle-tape-head.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:02 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/interpack-tape-head-replacement-roller-shell.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:03 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/5-5-inch-orange-bar-invoice-envelope.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/rear-rubber-roller-fj-40-03-for-eagle-tape-head.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:04 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/6-in-styled-packing-list-envelope.html returned 404 status code. 2025-11-08 13:04:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/plastic-material-heat-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:04 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/cac51-loveshaw-tape-cartridge.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:05 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/6-x-15-gold-backed-vacuum-pouch.html returned 404 status code. 2025-11-08 13:04:05 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-control-board-dbc2000e2013.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/28-x-24-x-60-case-packed-gusseted-bag-1-5-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/service-parts-8-inch-seal-cut-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/heat-seal-and-cut-machine.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/bestpack-2-inch-case-sealer-tape-head.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:06 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/12-inch-table-top-impulse-plastic-bag-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:07 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/plastic-material-heat-sealer.html returned 404 status code. 2025-11-08 13:04:08 [scrapy.extensions.logstats] (PID: 112) INFO: Crawled 5854 pages (at 339 pages/min), scraped 2734 items (at 122 items/min) 2025-11-08 13:04:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/30-x-18-x-48-case-packed-gusseted-bag-3-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:08 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/combi-2ez-sb-case-erector-with-bottom-sealer.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/drive-belt-for-intertape-2020-side-belt-and-top-belt-case-sealers.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eagle-main-roller-fg-03-02.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:09 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/pres-on-p4212-double-coated-polyethylene-foam-tape-52-x-54.html returned 404 status code. 2025-11-08 13:04:09 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/atp-red-vinyl-tape-1-inch-36-yards.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:10 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/2-x-2-x-36-120-edge-protector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:11 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/4-x-100-clear-poly-sheeting-6-mil.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:11 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/survivor-cut-resistant-gloves-7.html returned 404 status code. 2025-11-08 13:04:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/appleton.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:12 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-sprocket-spk-0050.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:12 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/welcome-to-wisconsin returned 404 status code. 2025-11-08 13:04:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/loveshaw-drive-belt-ldw-0057b-4.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:13 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/ideas/ebooks/protective-packaging-guide.html returned 404 status code. 2025-11-08 13:04:13 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/eaglehub.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:14 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/47-mailing-tube-box.html returned 404 status code. 2025-11-08 13:04:14 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/contact-us/appleton.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-june already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/blog/post/commitment-quality-brc-certification returned 404 status code. 2025-11-08 13:04:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/unraveling-the-history-of-tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/wexxar-bel-505-semi-automatic-case-erector.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/this-month-in-packaging-october already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:15 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/eaglehub.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/eaglehub.html landed on page that is not a product page. 2025-11-08 13:04:15 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/introduction-to-industrial-hot-melt-adhesives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/golden-harvest-grocery-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-june) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-june landed on page that is not a product page. 2025-11-08 13:04:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/unraveling-the-history-of-tape) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/unraveling-the-history-of-tape landed on page that is not a product page. 2025-11-08 13:04:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/this-month-in-packaging-october) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-october landed on page that is not a product page. 2025-11-08 13:04:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/contact-us/appleton.html) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/contact-us/appleton.html landed on page that is not a product page. 2025-11-08 13:04:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/introduction-to-industrial-hot-melt-adhesives) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/introduction-to-industrial-hot-melt-adhesives landed on page that is not a product page. 2025-11-08 13:04:16 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/blog/post/get-an-edge-on-protection already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:16 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input result = method(response=response, spider=spider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/middlewares.py", line 234, in process_spider_input raise ProductNotFound(f"Page {response.url} returned 404 status code.") scraping_utils.common.exceptions.ProductNotFound: Page https://www.rocketindustrial.com/13-chemical-resistant-gloves-s.html returned 404 status code. 2025-11-08 13:04:17 [HeadersSpooferDownloaderMiddleware] (PID: 112) WARNING: Request https://www.rocketindustrial.com/golden-harvest-grocery-bags.html already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2025-11-08 13:04:17 [scrapy.core.scraper] (PID: 112) ERROR: Spider error processing (referer: https://www.rocketindustrial.com/blog/post/get-an-edge-on-protection) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 162, in parse_product async for item in page.get_items(): ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 90, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.rocketindustrial.com/blog/post/get-an-edge-on-protection landed on page that is not a product page. 2025-11-08 13:04:18 [scrapy.downloadermiddlewares.retry] (PID: 112) ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2025-11-08 13:04:18 [scrapy.spidermiddlewares.httperror] (PID: 112) INFO: Ignoring response <500 https://www.rocketindustrial.com/golden-harvest-grocery-bags.html>: HTTP status code is not handled or not allowed 2025-11-08 13:04:18 [scrapy.core.engine] (PID: 112) INFO: Closing spider (finished) 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] ------------------------------ MONITORS ------------------------------ 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Extracted Items Monitor/test_stat_monitor... OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... SKIPPED (Unable to find 'spidermon/validation/fields/errors' in job stats.) 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Error Count Monitor/test_stat_monitor... FAIL 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Warning Count Monitor/test_stat_monitor... FAIL 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Unwanted HTTP codes monitor/Should not hit the limit of unwanted http status... FAIL 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Field Coverage Monitor/test_check_if_field_coverage_rules_are_met... OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Retry Count monitor/Should not hit the limit of requests that reached the maximum retry amount... OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Downloader Exceptions monitor/test_stat_monitor... SKIPPED (Unable to find 'downloader/exception_count' in job stats.) 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Successful Requests monitor/Should have at least the minimum number of successful requests... OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] Total Requests monitor/Should not hit the total limit of requests... OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) ERROR: [Spidermon] ====================================================================== FAIL: Error Count Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'log_count/ERROR' to be '<=' to '450.0'. Current value: '499' 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) ERROR: [Spidermon] ====================================================================== FAIL: Warning Count Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'log_count/WARNING' to be '<=' to '1000.0'. Current value: '3044' 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) ERROR: [Spidermon] ====================================================================== FAIL: Unwanted HTTP codes monitor/Should not hit the limit of unwanted http status ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/monitors.py", line 236, in test_check_unwanted_http_codes self.assertTrue(count <= max_errors, msg=msg) AssertionError: Found 149 Responses with status code=429 - This exceeds the limit of 120 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] 11 monitors in 0.005s 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] FAILED (failures=3, skipped=2) 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] -------------------------- FINISHED ACTIONS -------------------------- 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] 0 actions in 0.000s 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] --------------------------- PASSED ACTIONS --------------------------- 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] 0 actions in 0.000s 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] --------------------------- FAILED ACTIONS --------------------------- 2025-11-08 13:04:18 [spidermon.contrib.actions.slack] (PID: 112) WARNING: bot cannot finder user in slack org member list - default icon url used 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] CustomTemplateSendSlackMessageSpiderFinished... OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] ---------------------------------------------------------------------- 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] 1 action in 0.417s 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: [Spidermon] OK 2025-11-08 13:04:18 [rocket_industrial] (PID: 112) INFO: 289 URLs returned ProductNotFound. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/1-2-x-12-static-control-bubble-packaging.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/10-x-12-black-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/10-x-12-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/10-x-15-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/10-x-22-flairpak-400.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/12-inch-clear-hand-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/12-inch-stretch-film.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/12-x-22-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/13-chemical-resistant-gloves-s.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/15-5-x-9-875-curby-mailer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/15-inch-extended-core-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/15-inch-hand-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/17-inch-amtopp-pallet-lock-38-gauge-hand-stretch-film-1476-foot-rolls.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/18-5-x-11-875-curby-mailer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/18-inch-cast-hand-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/18-inch-green-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/18-inch-red-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/18-x-23-1-2-self-seal-bubble-pouches.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-30-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-30-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-36-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-36-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-40-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-40-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-48-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-48-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-75-mailing-tube-end-caps.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-75-x-35-mailing-tubes.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-75-x-47-mailing-tubes.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-inch-x-1500-yard-packing-tape.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-12-120-edge-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-12-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-12-225-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-18-120-edge-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-18-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-18-225-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-24-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-24-225-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-30-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-30-225-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-40-225-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-48-vboard-edge-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-6-225-strapping-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-60-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-60-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/2-x-2-x-72-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/20-inch-amtopp-70-gauge-high-performance-stretch-wrap-6500-foot-rolls.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/20-inch-cold-force-machine-film-71-gauge-stretch-wrap-6500-foot-rolls.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/20-inch-extended-core-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/20-inch-extended-handle-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/20-inch-nexus-machine-film-80-gauge-stretch-wrap-6000-foot-rolls.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/20-x-800-goodwrappers-120-gauge-economy-hand-stretch-film.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-16-x-24-static-control-bubble-packaging.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-inch-clear-poly-tubing-2-mil.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-inch-interpack-standard-tape-head-1.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-12-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-12-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-18-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-18-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-24-100-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-24-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-30-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-30-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-36-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-40-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-48-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-6-160-strapping-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-6-225-strapping-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-60-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-60-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-60-25-edge-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-72-120-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3-x-3-x-72-160-edge-protector.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/30-inch-extended-core-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/30-inch-hand-held-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/35lb-vci-poly-coated-kraft-paper-roll-48-x-200yd.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3m-200a-case-sealer-rfb.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3m-5490-ptfe-extruded-film-tape-12-x-36.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3m-basic-tape-dispenser.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3m-deluxe-packing-tape-dispenser.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/3m-lane-marking-applicator.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/4-6-white-perforated-transfer-labels.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/4-x-4-x-2-160-strapping-protectors.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/4-x-7-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/4-x-9-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/40lb-vci-kraft-paper-roll-36-x-200yd.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/40lb-vci-kraft-paper-roll-48-x-200yd.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/47-mailing-tube-box.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/5-5-inch-orange-face-invoice-envelope.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/6-in-packing-list-enclosed-red-face.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/6-in-styled-packing-list-envelope.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/6-x-10-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/6-x-12-black-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/6-x-12-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/6-x-15-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/6-x-8-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/7-x-10-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/8-x-10-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/8-x-12-gold-backed-vacuum-pouch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/8-x-22-shrink-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/9-5-x-9-875-curby-mailer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/air-bubble-roll-3-16-48-750.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/atp-cvt-636-colored-vinyl-tape-logs.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/atp-dc-4420lb-double-coated-pvc-tape-39-x-60.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/atp-pgm-uv14-blue-painters-masking-tape-39-x-60.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/atp-sst-936l-black-yellow-striped-warning-tape-49-x-36.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/battery-powered-steel-strapping-sealer-tensionser-kits.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/bestpack-bg18-machine-length-tape-2-x-1000.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/bestpack-csx-automatic-random-case-sealer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/10-facts-about-packaging-that-will-impress-your-friends is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/100-anniversary-of-united-paper is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/2022-year-in-review is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/3m-products-close-by is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/4-manufacturers-going-zero-waste-to-landfill is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/5-uses-vr-manufacturing is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/amazon-packaging-waste is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/amazons-new-machines-pack-five-times-faster-than-humans is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/animals-in-packaged-food is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/automation-road-map is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/breaking-down-blockchain-part-1 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/brewing-insights-tips-from-wisconsin-brewers is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/brown-is-green-story-of-corrugated-recycling-success is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/commitment-quality-brc-certification is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/coronavirus-and-packaging is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/creative-packaging-mattress-industry is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/driving-dramatic-growth-robotics is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/eco-viewing-our-favorite-nature-documentaries is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-aaron-stelzl is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-bob-schymanski is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-brian-garvin is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-jen-rybicki is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-josh-struck is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-katie-zoborowski is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-lauri is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/employee-spotlight-matthew-bruss is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/engineering-success-developing-future-packaging is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/factors-to-consider-before-automating is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/five-takeaways-spc-conference is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/greener-way-to-protect-products-during-shipment is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/happy-holidays-from-rocket-industrial is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/helping-the-heroes-of-covid19 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/history-of-the-n95-mask is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/how-to-choose-the-right-type-of-safety-gloves is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/how-to-properly-remove-disposable-gloves is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/influence-packaging-brand-image is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/interview-packaging-engineer is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/introducing-project100k is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/ipg-american-made-tapes-films-packaging is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/loops-reusable-packaging-program-launches-in-us is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/make-your-packaging-operation-a-great-place-to-work is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/matting-programs-are-a-winning-strategy-for-industrial-athletes is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/meet-inside-sales-representative is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/mile-of-music-mike-maimone is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/navigate-supply-shortages-for-packaging-materials is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/new-website-launch is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/no-end-labor-shortages is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/now-open-packlytics-packaging-test-lab is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/operation-packaging-care-2017 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/our-response-to-current-market-challenges is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/pack-to-the-future-packaging-predictions is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/packaging-automation-101-where-to-begin is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/packaging-plays-vital-role-covid-19-vaccine is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/packaging-tips-for-new-businesses-and-startups is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/packaging-wrap-up-january-2021 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/plastic-trash-turned-to-art is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/preparing-workplaces-beyond-covid-19 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/reinvigorating-american-manufacturing is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/remote-monitoring-becoming-essential-for-many-operations is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/right-level-of-automation is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/robot-vs-cobot is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/robots-automation-take-jobs is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/robots-tech-take-over-pyeongchang is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/should-i-consider-a-used-stretch-wrap-machine is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/small-footpring-big-savings-from-a-palletizer is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/social-distancing-in-manufacturing-will-spike-trend-adoption is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/splice-up-your-life-with-our-top-selling-splicing-tapes is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/start-at-the-end-to-understand-the-beginning is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/stretch-wrapper-advantage is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/super-bowl-going-plastic-free is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/sustainability-in-the-ecommerce-world is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-february-2021 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/this-month-in-packaging-march-2021 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/war-on-waste-what-you-need-to-know is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/welcome-to-wisconsin is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/were-in-this-together-rockets-covid19-statement is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/will-germanys-new-packaging-law-impact-you is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/year-in-review-2021 is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/blog/post/you-shouldnt-be-totally-afraid-of-automation is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/combi-ce-10-carton-erector-bottom-case-sealer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/combi-replacement-blades.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/create-your-own-job.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/eagle-q31-19-battery-powered-strapping-tool.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/eagle-q31-battery-powered-strapping-tool.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ebooks.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ep-820-ball-knob-dispenser.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/fiberglass-cloth-tape-42-x-36-yards-7-5-mil.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/general-purpose-poly-strapping-kit.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/good-natured-70-gauge-hand-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/good-natured-70-gauge-machine-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/good-natured-80-gauge-hand-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/good-natured-80-gauge-machine-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/hand-saver-dispenser-w-tape.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/how-to-load-a-tape-dispenser.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ideas/ebooks/brewers-packaging-guide.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ideas/ebooks/cold-chain-packaging.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ideas/ebooks/packaging-automation-ebook.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ideas/ebooks/protective-packaging-guide.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ideas/ebooks/strapping-banding-ebook.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/intertape-400-medium-duty-clear-packaging-tape-2-x-110-yards.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-2-x-1500-yards-1-6-mil.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-3-x-1500-yards-1-6-mil.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/intertape-8100-heavy-duty-clear-machine-length-packaging-tape-2-x-1500-yards-2-2-mil.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/intertape-8100-heavy-duty-clear-machine-length-packaging-tape-3-x-1500-yards-2-2-mil.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ipg-591-flatback-paper-tape-23-x-36-12-mil.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ipg-bt100-blast-tape-log.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ipg-pg49-paper-masking-tape-59-x-60.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ipg-stretchflex-sf1-100-gauge-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ipg-stretchflex-sf1-80-gauge-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ipg-stretchflex-sf1-90-gauge-stretch-wrap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/jumbo-general-purpose-strapping-kit.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/jumbo-postal-approved-strapping-kit.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/microjet-conditioner-fluid-640.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/nelson-wrap-dispenser.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/nwd-betterwrapper.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/nwd-littlenelson.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/orgapack-200-battery.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/our-capabilities.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/our-promise.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/packaging-testing.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/packlytics-semi-automatic-smart-stretch-wrapper.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/packlytics-stretch-wrap-monitoring-system.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/pink-amine-free-sealable-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/pink-resealable-antistatic-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/plastic-material-heat-sealer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/polyair-airspace-air-pillow-film-8-x-5-inch-roll.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/polychem-reg-b800-battery-powered-strapping-tool.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/polyken-105c-double-coated-cloth-tape-60-x-25.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/polyken-510-red-gaffers-tape-56-x-55.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/postal-approved-poly-strapping-kit.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/pres-on-p4212-double-coated-polyethylene-foam-tape-52-x-54.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/quad-701-fast-set-hot-melt-glue-stick.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/quad-725-general-purpose-hot-melt-glue-stick.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/refresh-azure-foam-soap.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/resealable-antistatic-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/resealable-pink-antistatic-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/resealable-static-free-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/resealable-transparent-antistatic-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/ri-200-30-day-guarantee.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/rocket-industrial-180-high-temp-hot-melt-glue-gun-1-2-inch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/rocket-industrial-low-temp-hot-melt-glue-gun.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/safety-aisle-tape-applicator.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/saint-gobain-200a-silicone-sponge-tape-36-x-10.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-control-board.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-drive-wheel.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-feed-switch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-holding-gripper.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-k19-vbelt.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-microswitch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-power-switch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-reset-switch.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-smoke-fan-110v.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/samuel-tension-spring.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/scapa-136-white-polyethylene-film-tape-50-x-60.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/selecting-a-case-sealer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/service-kit-4-inch-plastic-heat-sealer.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/static-protected-resealable-bag.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/steel-box-cutter-utility-knife.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/storopack-paperbubble.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/strapping-pallet-feeder-tool.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-100-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-115-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-50-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-55-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-60-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-70-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-75-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-90-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/stretchflex-stretch-wrap-80-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/superflex-39-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/superflex-42-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/superflex-45-gauge.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/superflex-63-gauge-long-roll.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/survivor-cut-resistant-gloves-13.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/survivor-cut-resistant-gloves-7.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/synergy-low-profile-turntable-demo.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/tach-it-3560-semi-automatic-twist-tie-machine.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/taconic-6085-06-ptfe-fiberglass-cloth-tape-39-x-36.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/taconic-6115-03-skived-ptfe-tape-36-x-36.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URL https://www.rocketindustrial.com/taconic-6445-05-high-modulus-ptfe-film-tape-24-x-36.html is already flagged as 'DISABLED_NOT_FOUND' in the catalog_urls table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) WARNING: 289 URLs were not found in the `catalog_urls` table. 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: URLs not found and not flagged: {'https://www.rocketindustrial.com/ideas/ebooks/brewers-packaging-guide.html', 'https://www.rocketindustrial.com/survivor-cut-resistant-gloves-13.html', 'https://www.rocketindustrial.com/3-x-3-x-30-160-edge-protector.html', 'https://www.rocketindustrial.com/20-inch-extended-handle-stretch-wrap.html', 'https://www.rocketindustrial.com/blog/post/sustainability-in-the-ecommerce-world', 'https://www.rocketindustrial.com/good-natured-80-gauge-hand-stretch-wrap.html', 'https://www.rocketindustrial.com/blog/post/matting-programs-are-a-winning-strategy-for-industrial-athletes', 'https://www.rocketindustrial.com/quad-701-fast-set-hot-melt-glue-stick.html', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-lauri', 'https://www.rocketindustrial.com/ideas/ebooks/strapping-banding-ebook.html', 'https://www.rocketindustrial.com/blog/post/happy-holidays-from-rocket-industrial', 'https://www.rocketindustrial.com/ipg-stretchflex-sf1-90-gauge-stretch-wrap.html', 'https://www.rocketindustrial.com/selecting-a-case-sealer.html', 'https://www.rocketindustrial.com/blog/post/engineering-success-developing-future-packaging', 'https://www.rocketindustrial.com/plastic-material-heat-sealer.html', 'https://www.rocketindustrial.com/blog/post/remote-monitoring-becoming-essential-for-many-operations', 'https://www.rocketindustrial.com/blog/post/interview-packaging-engineer', 'https://www.rocketindustrial.com/15-inch-extended-core-stretch-wrap.html', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-bob-schymanski', 'https://www.rocketindustrial.com/2-x-2-x-30-120-edge-protector.html', 'https://www.rocketindustrial.com/blog/post/how-to-choose-the-right-type-of-safety-gloves', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-40-160-edge-protector.html', 'https://www.rocketindustrial.com/intertape-8100-heavy-duty-clear-machine-length-packaging-tape-2-x-1500-yards-2-2-mil.html', 'https://www.rocketindustrial.com/20-inch-extended-core-stretch-wrap.html', 'https://www.rocketindustrial.com/microjet-conditioner-fluid-640.html', 'https://www.rocketindustrial.com/service-kit-4-inch-plastic-heat-sealer.html', 'https://www.rocketindustrial.com/2-x-2-x-12-225-edge-protector.html', 'https://www.rocketindustrial.com/static-protected-resealable-bag.html', 'https://www.rocketindustrial.com/7-x-10-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/3-x-3-x-72-120-edge-protector.html', 'https://www.rocketindustrial.com/blog/post/new-website-launch', 'https://www.rocketindustrial.com/samuel-feed-switch.html', 'https://www.rocketindustrial.com/tach-it-3560-semi-automatic-twist-tie-machine.html', 'https://www.rocketindustrial.com/ri-200-30-day-guarantee.html', 'https://www.rocketindustrial.com/polyair-airspace-air-pillow-film-8-x-5-inch-roll.html', 'https://www.rocketindustrial.com/3-x-3-x-60-120-edge-protector.html', 'https://www.rocketindustrial.com/atp-dc-4420lb-double-coated-pvc-tape-39-x-60.html', 'https://www.rocketindustrial.com/stretchflex-60-gauge.html', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-48-120-edge-protector.html', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-30-120-edge-protector.html', 'https://www.rocketindustrial.com/packlytics-stretch-wrap-monitoring-system.html', 'https://www.rocketindustrial.com/refresh-azure-foam-soap.html', 'https://www.rocketindustrial.com/good-natured-80-gauge-machine-stretch-wrap.html', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-katie-zoborowski', 'https://www.rocketindustrial.com/blog/post/robots-automation-take-jobs', 'https://www.rocketindustrial.com/packaging-testing.html', 'https://www.rocketindustrial.com/blog/post/brown-is-green-story-of-corrugated-recycling-success', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-48-160-edge-protector.html', 'https://www.rocketindustrial.com/create-your-own-job.html', 'https://www.rocketindustrial.com/13-chemical-resistant-gloves-s.html', 'https://www.rocketindustrial.com/8-x-12-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/blog/post/packaging-plays-vital-role-covid-19-vaccine', 'https://www.rocketindustrial.com/samuel-microswitch.html', 'https://www.rocketindustrial.com/8-x-22-shrink-bag.html', 'https://www.rocketindustrial.com/2-x-2-x-18-160-edge-protector.html', 'https://www.rocketindustrial.com/intertape-8100-heavy-duty-clear-machine-length-packaging-tape-3-x-1500-yards-2-2-mil.html', 'https://www.rocketindustrial.com/18-inch-red-stretch-wrap.html', 'https://www.rocketindustrial.com/4-x-7-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/blog/post/introducing-project100k', 'https://www.rocketindustrial.com/packlytics-semi-automatic-smart-stretch-wrapper.html', 'https://www.rocketindustrial.com/nwd-littlenelson.html', 'https://www.rocketindustrial.com/3m-5490-ptfe-extruded-film-tape-12-x-36.html', 'https://www.rocketindustrial.com/stretchflex-75-gauge.html', 'https://www.rocketindustrial.com/blog/post/influence-packaging-brand-image', 'https://www.rocketindustrial.com/ipg-bt100-blast-tape-log.html', 'https://www.rocketindustrial.com/bestpack-csx-automatic-random-case-sealer.html', 'https://www.rocketindustrial.com/blog/post/five-takeaways-spc-conference', 'https://www.rocketindustrial.com/3-x-3-x-40-120-edge-protector.html', 'https://www.rocketindustrial.com/8-x-10-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/2-x-2-x-60-120-edge-protector.html', 'https://www.rocketindustrial.com/saint-gobain-200a-silicone-sponge-tape-36-x-10.html', 'https://www.rocketindustrial.com/blog/post/amazon-packaging-waste', 'https://www.rocketindustrial.com/35lb-vci-poly-coated-kraft-paper-roll-48-x-200yd.html', 'https://www.rocketindustrial.com/blog/post/were-in-this-together-rockets-covid19-statement', 'https://www.rocketindustrial.com/blog/post/plastic-trash-turned-to-art', 'https://www.rocketindustrial.com/blog/post/our-response-to-current-market-challenges', 'https://www.rocketindustrial.com/ipg-stretchflex-sf1-100-gauge-stretch-wrap.html', 'https://www.rocketindustrial.com/blog/post/robot-vs-cobot', 'https://www.rocketindustrial.com/stretchflex-55-gauge.html', 'https://www.rocketindustrial.com/4-x-4-x-2-160-strapping-protectors.html', 'https://www.rocketindustrial.com/synergy-low-profile-turntable-demo.html', 'https://www.rocketindustrial.com/orgapack-200-battery.html', 'https://www.rocketindustrial.com/2-x-2-x-18-120-edge-protectors.html', 'https://www.rocketindustrial.com/3-x-3-x-60-160-edge-protector.html', 'https://www.rocketindustrial.com/superflex-42-gauge.html', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-aaron-stelzl', 'https://www.rocketindustrial.com/jumbo-postal-approved-strapping-kit.html', 'https://www.rocketindustrial.com/2-x-2-x-60-160-edge-protector.html', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-40-120-edge-protector.html', 'https://www.rocketindustrial.com/resealable-transparent-antistatic-bag.html', 'https://www.rocketindustrial.com/ideas/ebooks/packaging-automation-ebook.html', 'https://www.rocketindustrial.com/40lb-vci-kraft-paper-roll-36-x-200yd.html', 'https://www.rocketindustrial.com/2-x-2-x-48-vboard-edge-protectors.html', 'https://www.rocketindustrial.com/3m-deluxe-packing-tape-dispenser.html', 'https://www.rocketindustrial.com/blog/post/amazons-new-machines-pack-five-times-faster-than-humans', 'https://www.rocketindustrial.com/fiberglass-cloth-tape-42-x-36-yards-7-5-mil.html', 'https://www.rocketindustrial.com/12-inch-clear-hand-wrap.html', 'https://www.rocketindustrial.com/samuel-power-switch.html', 'https://www.rocketindustrial.com/stretchflex-100-gauge.html', 'https://www.rocketindustrial.com/6-in-styled-packing-list-envelope.html', 'https://www.rocketindustrial.com/blog/post/war-on-waste-what-you-need-to-know', 'https://www.rocketindustrial.com/3-x-3-x-6-160-strapping-protectors.html', 'https://www.rocketindustrial.com/blog/post/stretch-wrapper-advantage', 'https://www.rocketindustrial.com/3-x-3-x-60-25-edge-protectors.html', 'https://www.rocketindustrial.com/strapping-pallet-feeder-tool.html', 'https://www.rocketindustrial.com/polychem-reg-b800-battery-powered-strapping-tool.html', 'https://www.rocketindustrial.com/how-to-load-a-tape-dispenser.html', 'https://www.rocketindustrial.com/blog/post/packaging-automation-101-where-to-begin', 'https://www.rocketindustrial.com/40lb-vci-kraft-paper-roll-48-x-200yd.html', 'https://www.rocketindustrial.com/3-x-3-x-12-120-edge-protector.html', 'https://www.rocketindustrial.com/blog/post/packaging-tips-for-new-businesses-and-startups', 'https://www.rocketindustrial.com/combi-ce-10-carton-erector-bottom-case-sealer.html', 'https://www.rocketindustrial.com/ipg-591-flatback-paper-tape-23-x-36-12-mil.html', 'https://www.rocketindustrial.com/5-5-inch-orange-face-invoice-envelope.html', 'https://www.rocketindustrial.com/atp-cvt-636-colored-vinyl-tape-logs.html', 'https://www.rocketindustrial.com/blog/post/history-of-the-n95-mask', 'https://www.rocketindustrial.com/2-x-2-x-72-160-edge-protector.html', 'https://www.rocketindustrial.com/eagle-q31-battery-powered-strapping-tool.html', 'https://www.rocketindustrial.com/good-natured-70-gauge-hand-stretch-wrap.html', 'https://www.rocketindustrial.com/polyken-105c-double-coated-cloth-tape-60-x-25.html', 'https://www.rocketindustrial.com/15-inch-hand-stretch-wrap.html', 'https://www.rocketindustrial.com/3-inch-interpack-standard-tape-head-1.html', 'https://www.rocketindustrial.com/blog/post/helping-the-heroes-of-covid19', 'https://www.rocketindustrial.com/blog/post/this-month-in-packaging-march-2021', 'https://www.rocketindustrial.com/3-x-3-x-18-160-edge-protector.html', 'https://www.rocketindustrial.com/2-x-2-x-18-225-edge-protector.html', 'https://www.rocketindustrial.com/18-5-x-11-875-curby-mailer.html', 'https://www.rocketindustrial.com/20-inch-nexus-machine-film-80-gauge-stretch-wrap-6000-foot-rolls.html', 'https://www.rocketindustrial.com/6-x-10-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/general-purpose-poly-strapping-kit.html', 'https://www.rocketindustrial.com/ep-820-ball-knob-dispenser.html', 'https://www.rocketindustrial.com/blog/post/this-month-in-packaging-february-2021', 'https://www.rocketindustrial.com/blog/post/ipg-american-made-tapes-films-packaging', 'https://www.rocketindustrial.com/3-x-3-x-72-160-edge-protector.html', 'https://www.rocketindustrial.com/2-inch-x-1500-yard-packing-tape.html', 'https://www.rocketindustrial.com/6-x-12-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/blog/post/operation-packaging-care-2017', 'https://www.rocketindustrial.com/3-inch-clear-poly-tubing-2-mil.html', 'https://www.rocketindustrial.com/bestpack-bg18-machine-length-tape-2-x-1000.html', 'https://www.rocketindustrial.com/taconic-6445-05-high-modulus-ptfe-film-tape-24-x-36.html', 'https://www.rocketindustrial.com/blog/post/10-facts-about-packaging-that-will-impress-your-friends', 'https://www.rocketindustrial.com/blog/post/5-uses-vr-manufacturing', 'https://www.rocketindustrial.com/3-x-3-x-48-120-edge-protector.html', 'https://www.rocketindustrial.com/survivor-cut-resistant-gloves-7.html', 'https://www.rocketindustrial.com/2-75-x-47-mailing-tubes.html', 'https://www.rocketindustrial.com/10-x-12-black-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/3-x-3-x-6-225-strapping-protectors.html', 'https://www.rocketindustrial.com/6-x-12-black-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/blog/post/loops-reusable-packaging-program-launches-in-us', 'https://www.rocketindustrial.com/superflex-39-gauge.html', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-josh-struck', 'https://www.rocketindustrial.com/blog/post/no-end-labor-shortages', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-36-160-edge-protector.html', 'https://www.rocketindustrial.com/47-mailing-tube-box.html', 'https://www.rocketindustrial.com/blog/post/meet-inside-sales-representative', 'https://www.rocketindustrial.com/nelson-wrap-dispenser.html', 'https://www.rocketindustrial.com/18-x-23-1-2-self-seal-bubble-pouches.html', 'https://www.rocketindustrial.com/2-x-2-x-24-225-edge-protector.html', 'https://www.rocketindustrial.com/blog/post/mile-of-music-mike-maimone', 'https://www.rocketindustrial.com/blog/post/robots-tech-take-over-pyeongchang', 'https://www.rocketindustrial.com/stretchflex-stretch-wrap-80-gauge.html', 'https://www.rocketindustrial.com/blog/post/automation-road-map', 'https://www.rocketindustrial.com/stretchflex-90-gauge.html', 'https://www.rocketindustrial.com/2-75-x-35-mailing-tubes.html', 'https://www.rocketindustrial.com/blog/post/make-your-packaging-operation-a-great-place-to-work', 'https://www.rocketindustrial.com/4-6-white-perforated-transfer-labels.html', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-brian-garvin', 'https://www.rocketindustrial.com/good-natured-70-gauge-machine-stretch-wrap.html', 'https://www.rocketindustrial.com/scapa-136-white-polyethylene-film-tape-50-x-60.html', 'https://www.rocketindustrial.com/blog/post/splice-up-your-life-with-our-top-selling-splicing-tapes', 'https://www.rocketindustrial.com/blog/post/right-level-of-automation', 'https://www.rocketindustrial.com/blog/post/greener-way-to-protect-products-during-shipment', 'https://www.rocketindustrial.com/blog/post/how-to-properly-remove-disposable-gloves', 'https://www.rocketindustrial.com/blog/post/social-distancing-in-manufacturing-will-spike-trend-adoption', 'https://www.rocketindustrial.com/3-x-3-x-24-100-edge-protector.html', 'https://www.rocketindustrial.com/9-5-x-9-875-curby-mailer.html', 'https://www.rocketindustrial.com/blog/post/pack-to-the-future-packaging-predictions', 'https://www.rocketindustrial.com/polyken-510-red-gaffers-tape-56-x-55.html', 'https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-2-x-1500-yards-1-6-mil.html', 'https://www.rocketindustrial.com/taconic-6115-03-skived-ptfe-tape-36-x-36.html', 'https://www.rocketindustrial.com/2-75-mailing-tube-end-caps.html', 'https://www.rocketindustrial.com/blog/post/driving-dramatic-growth-robotics', 'https://www.rocketindustrial.com/blog/post/factors-to-consider-before-automating', 'https://www.rocketindustrial.com/3-x-3-x-30-120-edge-protector.html', 'https://www.rocketindustrial.com/intertape-400-medium-duty-clear-packaging-tape-2-x-110-yards.html', 'https://www.rocketindustrial.com/3m-basic-tape-dispenser.html', 'https://www.rocketindustrial.com/jumbo-general-purpose-strapping-kit.html', 'https://www.rocketindustrial.com/stretchflex-50-gauge.html', 'https://www.rocketindustrial.com/17-inch-amtopp-pallet-lock-38-gauge-hand-stretch-film-1476-foot-rolls.html', 'https://www.rocketindustrial.com/blog/post/100-anniversary-of-united-paper', 'https://www.rocketindustrial.com/blog/post/will-germanys-new-packaging-law-impact-you', 'https://www.rocketindustrial.com/steel-box-cutter-utility-knife.html', 'https://www.rocketindustrial.com/intertape-6100-light-duty-clear-machine-length-packaging-tape-3-x-1500-yards-1-6-mil.html', 'https://www.rocketindustrial.com/ipg-stretchflex-sf1-80-gauge-stretch-wrap.html', 'https://www.rocketindustrial.com/blog/post/small-footpring-big-savings-from-a-palletizer', 'https://www.rocketindustrial.com/ipg-pg49-paper-masking-tape-59-x-60.html', 'https://www.rocketindustrial.com/blog/post/year-in-review-2021', 'https://www.rocketindustrial.com/blog/post/super-bowl-going-plastic-free', 'https://www.rocketindustrial.com/rocket-industrial-180-high-temp-hot-melt-glue-gun-1-2-inch.html', 'https://www.rocketindustrial.com/4-x-9-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/10-x-22-flairpak-400.html', 'https://www.rocketindustrial.com/18-inch-green-stretch-wrap.html', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-36-120-edge-protector.html', 'https://www.rocketindustrial.com/10-x-15-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/superflex-63-gauge-long-roll.html', 'https://www.rocketindustrial.com/2-x-2-x-12-120-edge-protectors.html', 'https://www.rocketindustrial.com/12-inch-stretch-film.html', 'https://www.rocketindustrial.com/resealable-static-free-bag.html', 'https://www.rocketindustrial.com/10-x-12-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/3m-200a-case-sealer-rfb.html', 'https://www.rocketindustrial.com/blog/post/brewing-insights-tips-from-wisconsin-brewers', 'https://www.rocketindustrial.com/3-x-3-x-36-160-edge-protector.html', 'https://www.rocketindustrial.com/20-inch-cold-force-machine-film-71-gauge-stretch-wrap-6500-foot-rolls.html', 'https://www.rocketindustrial.com/blog/post/4-manufacturers-going-zero-waste-to-landfill', 'https://www.rocketindustrial.com/stretchflex-115-gauge.html', 'https://www.rocketindustrial.com/rocket-industrial-low-temp-hot-melt-glue-gun.html', 'https://www.rocketindustrial.com/blog/post/welcome-to-wisconsin', 'https://www.rocketindustrial.com/postal-approved-poly-strapping-kit.html', 'https://www.rocketindustrial.com/blog/post/start-at-the-end-to-understand-the-beginning', 'https://www.rocketindustrial.com/our-promise.html', 'https://www.rocketindustrial.com/atp-pgm-uv14-blue-painters-masking-tape-39-x-60.html', 'https://www.rocketindustrial.com/combi-replacement-blades.html', 'https://www.rocketindustrial.com/2-1-2-x-2-1-2-x-30-160-edge-protector.html', 'https://www.rocketindustrial.com/blog/post/2022-year-in-review', 'https://www.rocketindustrial.com/blog/post/reinvigorating-american-manufacturing', 'https://www.rocketindustrial.com/superflex-45-gauge.html', 'https://www.rocketindustrial.com/stretchflex-70-gauge.html', 'https://www.rocketindustrial.com/pres-on-p4212-double-coated-polyethylene-foam-tape-52-x-54.html', 'https://www.rocketindustrial.com/20-inch-amtopp-70-gauge-high-performance-stretch-wrap-6500-foot-rolls.html', 'https://www.rocketindustrial.com/blog/post/animals-in-packaged-food', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-jen-rybicki', 'https://www.rocketindustrial.com/resealable-pink-antistatic-bag.html', 'https://www.rocketindustrial.com/2-x-2-x-40-225-edge-protector.html', 'https://www.rocketindustrial.com/taconic-6085-06-ptfe-fiberglass-cloth-tape-39-x-36.html', 'https://www.rocketindustrial.com/ebooks.html', 'https://www.rocketindustrial.com/pink-amine-free-sealable-bag.html', 'https://www.rocketindustrial.com/blog/post/should-i-consider-a-used-stretch-wrap-machine', 'https://www.rocketindustrial.com/3m-lane-marking-applicator.html', 'https://www.rocketindustrial.com/blog/post/creative-packaging-mattress-industry', 'https://www.rocketindustrial.com/blog/post/packaging-wrap-up-january-2021', 'https://www.rocketindustrial.com/samuel-smoke-fan-110v.html', 'https://www.rocketindustrial.com/samuel-control-board.html', 'https://www.rocketindustrial.com/3-x-3-x-24-120-edge-protector.html', 'https://www.rocketindustrial.com/battery-powered-steel-strapping-sealer-tensionser-kits.html', 'https://www.rocketindustrial.com/18-inch-cast-hand-stretch-wrap.html', 'https://www.rocketindustrial.com/6-x-15-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/blog/post/employee-spotlight-matthew-bruss', 'https://www.rocketindustrial.com/blog/post/coronavirus-and-packaging', 'https://www.rocketindustrial.com/nwd-betterwrapper.html', 'https://www.rocketindustrial.com/30-inch-hand-held-stretch-wrap.html', 'https://www.rocketindustrial.com/hand-saver-dispenser-w-tape.html', 'https://www.rocketindustrial.com/blog/post/3m-products-close-by', 'https://www.rocketindustrial.com/30-inch-extended-core-stretch-wrap.html', 'https://www.rocketindustrial.com/atp-sst-936l-black-yellow-striped-warning-tape-49-x-36.html', 'https://www.rocketindustrial.com/2-x-2-x-30-225-edge-protector.html', 'https://www.rocketindustrial.com/blog/post/preparing-workplaces-beyond-covid-19', 'https://www.rocketindustrial.com/our-capabilities.html', 'https://www.rocketindustrial.com/quad-725-general-purpose-hot-melt-glue-stick.html', 'https://www.rocketindustrial.com/3-x-3-x-12-160-edge-protector.html', 'https://www.rocketindustrial.com/blog/post/eco-viewing-our-favorite-nature-documentaries', 'https://www.rocketindustrial.com/eagle-q31-19-battery-powered-strapping-tool.html', 'https://www.rocketindustrial.com/2-x-2-x-24-160-edge-protector.html', 'https://www.rocketindustrial.com/pink-resealable-antistatic-bag.html', 'https://www.rocketindustrial.com/blog/post/breaking-down-blockchain-part-1', 'https://www.rocketindustrial.com/storopack-paperbubble.html', 'https://www.rocketindustrial.com/6-x-8-gold-backed-vacuum-pouch.html', 'https://www.rocketindustrial.com/samuel-drive-wheel.html', 'https://www.rocketindustrial.com/samuel-reset-switch.html', 'https://www.rocketindustrial.com/blog/post/commitment-quality-brc-certification', 'https://www.rocketindustrial.com/20-x-800-goodwrappers-120-gauge-economy-hand-stretch-film.html', 'https://www.rocketindustrial.com/blog/post/now-open-packlytics-packaging-test-lab', 'https://www.rocketindustrial.com/3-16-x-24-static-control-bubble-packaging.html', 'https://www.rocketindustrial.com/12-x-22-vacuum-pouch.html', 'https://www.rocketindustrial.com/samuel-tension-spring.html', 'https://www.rocketindustrial.com/resealable-antistatic-bag.html', 'https://www.rocketindustrial.com/blog/post/navigate-supply-shortages-for-packaging-materials', 'https://www.rocketindustrial.com/3-x-3-x-18-120-edge-protector.html', 'https://www.rocketindustrial.com/1-2-x-12-static-control-bubble-packaging.html', 'https://www.rocketindustrial.com/ideas/ebooks/protective-packaging-guide.html', 'https://www.rocketindustrial.com/samuel-holding-gripper.html', 'https://www.rocketindustrial.com/air-bubble-roll-3-16-48-750.html', 'https://www.rocketindustrial.com/samuel-k19-vbelt.html', 'https://www.rocketindustrial.com/ideas/ebooks/cold-chain-packaging.html', 'https://www.rocketindustrial.com/safety-aisle-tape-applicator.html', 'https://www.rocketindustrial.com/2-x-2-x-12-160-edge-protector.html', 'https://www.rocketindustrial.com/2-x-2-x-6-225-strapping-protectors.html', 'https://www.rocketindustrial.com/blog/post/you-shouldnt-be-totally-afraid-of-automation', 'https://www.rocketindustrial.com/15-5-x-9-875-curby-mailer.html', 'https://www.rocketindustrial.com/6-in-packing-list-enclosed-red-face.html'} 2025-11-08 13:04:20 [rocket_industrial] (PID: 112) INFO: Finished processing 'not found' URLs in the `catalog_urls` table. 2025-11-08 13:04:21 [scrapy.extensions.feedexport] (PID: 112) INFO: Stored bq feed (750 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:3 2025-11-08 13:04:21 [scrapy.statscollectors] (PID: 112) INFO: Dumping Scrapy stats: {'HeadersSpooferDownloaderMiddleware/spoofed': 6097, 'NotFoundHandlerSpiderMiddleware/HttpError': 12, 'NotFoundHandlerSpiderMiddleware/NotProductPage': 796, 'NotFoundHandlerSpiderMiddleware/ProductNotFound': 289, 'NotFoundHandlerSpiderMiddleware/not_found/404_response': 289, 'NotFoundHandlerSpiderMiddleware/not_found/ignored': 289, 'NotFoundHandlerSpiderMiddleware/not_found/retrieved': 289, 'big_query/url': 3057, 'downloader/request_bytes': 5042969, 'downloader/request_count': 6097, 'downloader/request_method_count/GET': 6097, 'downloader/response_bytes': 424791043, 'downloader/response_count': 6097, 'downloader/response_status_count/200': 5603, 'downloader/response_status_count/301': 21, 'downloader/response_status_count/302': 6, 'downloader/response_status_count/403': 2, 'downloader/response_status_count/404': 289, 'downloader/response_status_count/429': 149, 'downloader/response_status_count/500': 27, 'dupefilter/filtered': 736, 'elapsed_time_seconds': 1150.21127, 'feedexport/success_count/BigQueryFeedStorage': 3, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2025, 11, 8, 13, 4, 18, 522343, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 1949900800, 'httpcompression/response_count': 5894, 'httperror/response_ignored_count': 12, 'httperror/response_ignored_status_count/403': 2, 'httperror/response_ignored_status_count/429': 1, 'httperror/response_ignored_status_count/500': 9, 'item_scraped_count': 2750, 'items_per_minute': None, 'log_count/ERROR': 502, 'log_count/INFO': 368, 'log_count/WARNING': 3046, 'memusage/max': 253448192, 'memusage/startup': 125988864, 'poet/injector/catalog_extraction.pages.rocket_industrial.RocketIndustrialProductsPageObject': 2832, 'product_status/not_available_online': 203, 'proxy_manager/ignored/proxy_defined': 193, 'proxy_manager/processed': 5904, 'request_depth_max': 5, 'response_received_count': 5904, 'responses_per_minute': None, 'retry/count': 166, 'retry/max_reached': 10, 'retry/reason_count/429 Unknown Status': 148, 'retry/reason_count/500 Internal Server Error': 18, 'scheduler/dequeued': 6097, 'scheduler/dequeued/memory': 6097, 'scheduler/enqueued': 6097, 'scheduler/enqueued/memory': 6097, 'spider_exceptions/NotProductPage': 199, 'spider_exceptions/ProductNotFound': 289, 'spidermon/validation/fields': 57750, 'spidermon/validation/items': 2750, 'spidermon/validation/validators': 1, 'spidermon/validation/validators/item/jsonschema': True, 'spidermon_field_coverage/dict/brand': 1.0, 'spidermon_field_coverage/dict/categories': 1.0, 'spidermon_field_coverage/dict/countryOfOrigin': 1.0, 'spidermon_field_coverage/dict/description': 1.0, 'spidermon_field_coverage/dict/imageUrl': 1.0, 'spidermon_field_coverage/dict/inStock': 1.0, 'spidermon_field_coverage/dict/isFreeShipping': 1.0, 'spidermon_field_coverage/dict/leadTime': 1.0, 'spidermon_field_coverage/dict/manufacturer': 1.0, 'spidermon_field_coverage/dict/manufacturerSku': 1.0, 'spidermon_field_coverage/dict/name': 1.0, 'spidermon_field_coverage/dict/packagingIncrement': 1.0, 'spidermon_field_coverage/dict/prices': 1.0, 'spidermon_field_coverage/dict/productStatus': 1.0, 'spidermon_field_coverage/dict/relatedSkus': 1.0, 'spidermon_field_coverage/dict/specifications': 1.0, 'spidermon_field_coverage/dict/supplier': 1.0, 'spidermon_field_coverage/dict/supplierSku': 1.0, 'spidermon_field_coverage/dict/uom': 1.0, 'spidermon_field_coverage/dict/url': 1.0, 'spidermon_field_coverage/dict/weight': 1.0, 'spidermon_item_scraped_count': 2750, 'spidermon_item_scraped_count/dict': 2750, 'spidermon_item_scraped_count/dict/brand': 2750, 'spidermon_item_scraped_count/dict/categories': 2750, 'spidermon_item_scraped_count/dict/countryOfOrigin': 2750, 'spidermon_item_scraped_count/dict/description': 2750, 'spidermon_item_scraped_count/dict/imageUrl': 2750, 'spidermon_item_scraped_count/dict/inStock': 2750, 'spidermon_item_scraped_count/dict/isFreeShipping': 2750, 'spidermon_item_scraped_count/dict/leadTime': 2750, 'spidermon_item_scraped_count/dict/manufacturer': 2750, 'spidermon_item_scraped_count/dict/manufacturerSku': 2750, 'spidermon_item_scraped_count/dict/name': 2750, 'spidermon_item_scraped_count/dict/packagingIncrement': 2750, 'spidermon_item_scraped_count/dict/prices': 2750, 'spidermon_item_scraped_count/dict/productStatus': 2750, 'spidermon_item_scraped_count/dict/relatedSkus': 2750, 'spidermon_item_scraped_count/dict/specifications': 2750, 'spidermon_item_scraped_count/dict/supplier': 2750, 'spidermon_item_scraped_count/dict/supplierSku': 2750, 'spidermon_item_scraped_count/dict/uom': 2750, 'spidermon_item_scraped_count/dict/url': 2750, 'spidermon_item_scraped_count/dict/weight': 2750, 'start_requests/big_query': 3057, 'start_time': datetime.datetime(2025, 11, 8, 12, 45, 8, 311073, tzinfo=datetime.timezone.utc)} 2025-11-08 13:04:21 [scrapy.core.engine] (PID: 112) INFO: Spider closed (finished)